Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/schnet_feat_avg_feats_mpnn_from_auc/config.json --data_path /home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/schnet_feat_avg_feats_mpnn_from_auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=1),
 'dropout': 0.2,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat_avg/train_auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 2100,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 2100,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'auc',
 'metrics': ['auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/schnet_feat_avg_feats_mpnn_from_auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat_avg/test_auc.npz'],
 'separate_test_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat_avg/val_auc.npz'],
 'separate_val_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=2100, out_features=2100, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.2, inplace=False)
    (7): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 18,866,401
Moving model to cuda
Epoch 0
Validation auc = 0.737089
Epoch 1
Validation auc = 0.703756
Epoch 2
Validation auc = 0.721127
Epoch 3
Validation auc = 0.731925
Epoch 4
Loss = 8.7747e-01, PNorm = 94.3522, GNorm = 17.6439, lr_0 = 6.5000e-04
Validation auc = 0.694366
Epoch 5
Validation auc = 0.635211
Epoch 6
Validation auc = 0.683099
Epoch 7
Validation auc = 0.714554
Epoch 8
Validation auc = 0.723005
Epoch 9
Loss = 8.3591e-01, PNorm = 94.6974, GNorm = 17.7230, lr_0 = 9.9743e-04
Validation auc = 0.625352
Epoch 10
Validation auc = 0.621596
Epoch 11
Validation auc = 0.665258
Epoch 12
Validation auc = 0.701878
Epoch 13
Validation auc = 0.740376
Epoch 14
Loss = 5.1481e-01, PNorm = 95.0581, GNorm = 9.8032, lr_0 = 9.8890e-04
Validation auc = 0.728169
Epoch 15
Validation auc = 0.669014
Epoch 16
Validation auc = 0.669953
Epoch 17
Validation auc = 0.640845
Epoch 18
Validation auc = 0.680282
Epoch 19
Loss = 5.5054e-01, PNorm = 95.3048, GNorm = 11.4122, lr_0 = 9.8045e-04
Validation auc = 0.707981
Epoch 20
Validation auc = 0.715493
Epoch 21
Validation auc = 0.687793
Epoch 22
Validation auc = 0.642723
Epoch 23
Validation auc = 0.625352
Epoch 24
Loss = 2.9028e-01, PNorm = 95.4872, GNorm = 6.8045, lr_0 = 9.7207e-04
Validation auc = 0.663850
Epoch 25
Validation auc = 0.704225
Epoch 26
Validation auc = 0.725822
Epoch 27
Validation auc = 0.728169
Epoch 28
Validation auc = 0.721127
Epoch 29
Loss = 4.4926e-01, PNorm = 95.6488, GNorm = 3.3906, lr_0 = 9.6376e-04
Validation auc = 0.705634
Epoch 30
Validation auc = 0.683568
Epoch 31
Validation auc = 0.642254
Epoch 32
Validation auc = 0.659155
Epoch 33
Validation auc = 0.692019
Epoch 34
Loss = 3.4984e-01, PNorm = 95.8293, GNorm = 6.7686, lr_0 = 9.5552e-04
Validation auc = 0.708920
Epoch 35
Validation auc = 0.724413
Epoch 36
Validation auc = 0.723944
Epoch 37
Validation auc = 0.701408
Epoch 38
Validation auc = 0.697183
Epoch 39
Loss = 1.9107e-01, PNorm = 96.0674, GNorm = 3.3088, lr_0 = 9.4735e-04
Validation auc = 0.703756
Epoch 40
Validation auc = 0.718310
Epoch 41
Validation auc = 0.730516
Epoch 42
Validation auc = 0.732864
Epoch 43
Validation auc = 0.721596
Epoch 44
Loss = 2.5959e-01, PNorm = 96.2896, GNorm = 5.0447, lr_0 = 9.3925e-04
Validation auc = 0.707042
Epoch 45
Validation auc = 0.693427
Epoch 46
Validation auc = 0.692958
Epoch 47
Validation auc = 0.677465
Epoch 48
Validation auc = 0.693897
Epoch 49
Loss = 1.9314e-01, PNorm = 96.5308, GNorm = 3.2590, lr_0 = 9.3122e-04
Validation auc = 0.707042
Epoch 50
Validation auc = 0.688263
Epoch 51
Validation auc = 0.665258
Epoch 52
Validation auc = 0.663380
Epoch 53
Validation auc = 0.698122
Epoch 54
Loss = 2.2709e-01, PNorm = 96.7883, GNorm = 3.7723, lr_0 = 9.2326e-04
Validation auc = 0.718779
Epoch 55
Validation auc = 0.706103
Epoch 56
Validation auc = 0.707512
Epoch 57
Validation auc = 0.730986
Epoch 58
Validation auc = 0.739906
Epoch 59
Loss = 1.3551e-01, PNorm = 97.0736, GNorm = 2.4716, lr_0 = 9.1537e-04
Validation auc = 0.736150
Epoch 60
Validation auc = 0.723005
Epoch 61
Validation auc = 0.714085
Epoch 62
Validation auc = 0.727230
Epoch 63
Validation auc = 0.738028
Epoch 64
Loss = 8.5781e-02, PNorm = 97.3034, GNorm = 2.2651, lr_0 = 9.0754e-04
Validation auc = 0.738498
Epoch 65
Validation auc = 0.730516
Epoch 66
Validation auc = 0.705634
Epoch 67
Validation auc = 0.707981
Epoch 68
Validation auc = 0.706573
Epoch 69
Loss = 2.0244e-01, PNorm = 97.5345, GNorm = 4.8175, lr_0 = 8.9978e-04
Validation auc = 0.703286
Epoch 70
Validation auc = 0.689202
Epoch 71
Validation auc = 0.715962
Epoch 72
Validation auc = 0.718310
Epoch 73
Validation auc = 0.712676
Epoch 74
Loss = 6.3834e-02, PNorm = 97.8115, GNorm = 1.5539, lr_0 = 8.9209e-04
Validation auc = 0.716432
Epoch 75
Validation auc = 0.733803
Epoch 76
Validation auc = 0.744601
Epoch 77
Validation auc = 0.746479
Epoch 78
Validation auc = 0.743662
Epoch 79
Loss = 1.5766e-01, PNorm = 98.0867, GNorm = 0.8530, lr_0 = 8.8447e-04
Validation auc = 0.733333
Epoch 80
Validation auc = 0.734742
Epoch 81
Validation auc = 0.741315
Epoch 82
Validation auc = 0.746009
Epoch 83
Validation auc = 0.742254
Epoch 84
Loss = 1.5854e-01, PNorm = 98.2971, GNorm = 2.5885, lr_0 = 8.7691e-04
Validation auc = 0.727700
Epoch 85
Validation auc = 0.703756
Epoch 86
Validation auc = 0.695305
Epoch 87
Validation auc = 0.696244
Epoch 88
Validation auc = 0.694836
Epoch 89
Loss = 1.1992e-01, PNorm = 98.5536, GNorm = 2.4965, lr_0 = 8.6941e-04
Validation auc = 0.695775
Epoch 90
Validation auc = 0.700469
Epoch 91
Validation auc = 0.719718
Epoch 92
Validation auc = 0.739437
Epoch 93
Validation auc = 0.741315
Epoch 94
Loss = 1.9973e-01, PNorm = 98.7944, GNorm = 4.2012, lr_0 = 8.6198e-04
Validation auc = 0.728638
Epoch 95
Validation auc = 0.714554
Epoch 96
Validation auc = 0.674178
Epoch 97
Validation auc = 0.666667
Epoch 98
Validation auc = 0.687324
Epoch 99
Loss = 4.8592e-02, PNorm = 99.0350, GNorm = 1.3577, lr_0 = 8.5461e-04
Validation auc = 0.704695
Epoch 100
Validation auc = 0.713146
Epoch 101
Validation auc = 0.734272
Epoch 102
Validation auc = 0.743662
Epoch 103
Validation auc = 0.749296
Epoch 104
Loss = 1.2317e-01, PNorm = 99.2828, GNorm = 1.8858, lr_0 = 8.4730e-04
Validation auc = 0.738967
Epoch 105
Validation auc = 0.723944
Epoch 106
Validation auc = 0.715962
Epoch 107
Validation auc = 0.729108
Epoch 108
Validation auc = 0.740845
Epoch 109
Loss = 9.6303e-02, PNorm = 99.5457, GNorm = 3.9136, lr_0 = 8.4006e-04
Validation auc = 0.739906
Epoch 110
Validation auc = 0.730516
Epoch 111
Validation auc = 0.728169
Epoch 112
Validation auc = 0.733333
Epoch 113
Validation auc = 0.725822
Epoch 114
Loss = 9.4569e-02, PNorm = 99.8404, GNorm = 1.1453, lr_0 = 8.3288e-04
Validation auc = 0.706573
Epoch 115
Validation auc = 0.693427
Epoch 116
Validation auc = 0.693897
Epoch 117
Validation auc = 0.707512
Epoch 118
Validation auc = 0.718310
Epoch 119
Loss = 8.4613e-02, PNorm = 100.0963, GNorm = 2.6059, lr_0 = 8.2576e-04
Validation auc = 0.719249
Epoch 120
Validation auc = 0.729108
Epoch 121
Validation auc = 0.722535
Epoch 122
Validation auc = 0.728169
Epoch 123
Validation auc = 0.739437
Epoch 124
Loss = 8.2483e-02, PNorm = 100.2937, GNorm = 1.1460, lr_0 = 8.1870e-04
Validation auc = 0.740845
Epoch 125
Validation auc = 0.732394
Epoch 126
Validation auc = 0.730516
Epoch 127
Validation auc = 0.729577
Epoch 128
Validation auc = 0.728169
Epoch 129
Loss = 4.0614e-02, PNorm = 100.4452, GNorm = 0.3626, lr_0 = 8.1170e-04
Validation auc = 0.731455
Epoch 130
Validation auc = 0.727230
Epoch 131
Validation auc = 0.725822
Epoch 132
Validation auc = 0.724883
Epoch 133
Validation auc = 0.725822
Epoch 134
Loss = 4.3153e-02, PNorm = 100.5995, GNorm = 2.1689, lr_0 = 8.0476e-04
Validation auc = 0.728638
Epoch 135
Validation auc = 0.727700
Epoch 136
Validation auc = 0.731925
Epoch 137
Validation auc = 0.735211
Epoch 138
Validation auc = 0.740376
Epoch 139
Loss = 5.9150e-02, PNorm = 100.7485, GNorm = 2.0828, lr_0 = 7.9788e-04
Validation auc = 0.742723
Epoch 140
Validation auc = 0.744131
Epoch 141
Validation auc = 0.742254
Epoch 142
Validation auc = 0.736150
Epoch 143
Validation auc = 0.729108
Epoch 144
Loss = 1.0455e-01, PNorm = 100.9737, GNorm = 4.0858, lr_0 = 7.9106e-04
Validation auc = 0.729108
Epoch 145
Validation auc = 0.725822
Epoch 146
Validation auc = 0.728638
Epoch 147
Validation auc = 0.723474
Epoch 148
Validation auc = 0.721127
Epoch 149
Loss = 1.5401e-01, PNorm = 101.1799, GNorm = 3.3364, lr_0 = 7.8430e-04
Validation auc = 0.718779
Epoch 150
Validation auc = 0.721596
Epoch 151
Validation auc = 0.722066
Epoch 152
Validation auc = 0.728638
Epoch 153
Validation auc = 0.729108
Epoch 154
Loss = 1.2319e-01, PNorm = 101.3919, GNorm = 1.7624, lr_0 = 7.7759e-04
Validation auc = 0.730986
Epoch 155
Validation auc = 0.723005
Epoch 156
Validation auc = 0.722535
Epoch 157
Validation auc = 0.719249
Epoch 158
Validation auc = 0.715023
Epoch 159
Loss = 7.4196e-02, PNorm = 101.5472, GNorm = 2.7323, lr_0 = 7.7095e-04
Validation auc = 0.714554
Epoch 160
Validation auc = 0.711268
Epoch 161
Validation auc = 0.708920
Epoch 162
Validation auc = 0.707512
Epoch 163
Validation auc = 0.706573
Epoch 164
Loss = 1.3143e-01, PNorm = 101.6953, GNorm = 2.4046, lr_0 = 7.6436e-04
Validation auc = 0.709390
Epoch 165
Validation auc = 0.707512
Epoch 166
Validation auc = 0.710798
Epoch 167
Validation auc = 0.710798
Epoch 168
Validation auc = 0.701878
Epoch 169
Loss = 2.2529e-02, PNorm = 101.7977, GNorm = 0.5606, lr_0 = 7.5782e-04
Validation auc = 0.696714
Epoch 170
Validation auc = 0.699531
Epoch 171
Validation auc = 0.702817
Epoch 172
Validation auc = 0.700469
Epoch 173
Validation auc = 0.701878
Epoch 174
Loss = 1.6280e-01, PNorm = 101.9305, GNorm = 0.1781, lr_0 = 7.5134e-04
Validation auc = 0.703286
Epoch 175
Validation auc = 0.705634
Epoch 176
Validation auc = 0.709859
Epoch 177
Validation auc = 0.712676
Epoch 178
Validation auc = 0.707512
Epoch 179
Loss = 9.6817e-02, PNorm = 102.0777, GNorm = 2.5250, lr_0 = 7.4492e-04
Validation auc = 0.701408
Epoch 180
Validation auc = 0.696244
Epoch 181
Validation auc = 0.697653
Epoch 182
Validation auc = 0.686854
Epoch 183
Validation auc = 0.689671
Epoch 184
Loss = 7.6499e-02, PNorm = 102.2674, GNorm = 1.8358, lr_0 = 7.3855e-04
Validation auc = 0.690610
Epoch 185
Validation auc = 0.691080
Epoch 186
Validation auc = 0.691080
Epoch 187
Validation auc = 0.692958
Epoch 188
Validation auc = 0.696714
Epoch 189
Loss = 5.9161e-02, PNorm = 102.4807, GNorm = 2.3254, lr_0 = 7.3224e-04
Validation auc = 0.706573
Epoch 190
Validation auc = 0.709390
Epoch 191
Validation auc = 0.712676
Epoch 192
Validation auc = 0.715493
Epoch 193
Validation auc = 0.711268
Epoch 194
Loss = 4.8848e-02, PNorm = 102.6460, GNorm = 0.2947, lr_0 = 7.2598e-04
Validation auc = 0.709859
Epoch 195
Validation auc = 0.714085
Epoch 196
Validation auc = 0.714085
Epoch 197
Validation auc = 0.712676
Epoch 198
Validation auc = 0.710329
Epoch 199
Loss = 3.5546e-02, PNorm = 102.8067, GNorm = 1.4288, lr_0 = 7.1977e-04
Validation auc = 0.699531
Epoch 200
Validation auc = 0.694836
Epoch 201
Validation auc = 0.696244
Epoch 202
Validation auc = 0.694836
Epoch 203
Validation auc = 0.691549
Epoch 204
Loss = 9.8272e-03, PNorm = 102.9544, GNorm = 0.6982, lr_0 = 7.1362e-04
Validation auc = 0.688263
Epoch 205
Validation auc = 0.681221
Epoch 206
Validation auc = 0.682160
Epoch 207
Validation auc = 0.684507
Epoch 208
Validation auc = 0.696714
Epoch 209
Loss = 2.8342e-02, PNorm = 103.0889, GNorm = 0.2545, lr_0 = 7.0752e-04
Validation auc = 0.703286
Epoch 210
Validation auc = 0.709859
Epoch 211
Validation auc = 0.708451
Epoch 212
Validation auc = 0.703756
Epoch 213
Validation auc = 0.701408
Epoch 214
Loss = 7.8716e-03, PNorm = 103.2279, GNorm = 0.0245, lr_0 = 7.0147e-04
Validation auc = 0.697183
Epoch 215
Validation auc = 0.702817
Epoch 216
Validation auc = 0.708451
Epoch 217
Validation auc = 0.708920
Epoch 218
Validation auc = 0.704225
Epoch 219
Loss = 3.9696e-02, PNorm = 103.4084, GNorm = 0.6211, lr_0 = 6.9548e-04
Validation auc = 0.695305
Epoch 220
Validation auc = 0.680751
Epoch 221
Validation auc = 0.676056
Epoch 222
Validation auc = 0.677934
Epoch 223
Validation auc = 0.679812
Epoch 224
Loss = 3.6722e-02, PNorm = 103.5992, GNorm = 0.4020, lr_0 = 6.8953e-04
Validation auc = 0.683568
Epoch 225
Validation auc = 0.686854
Epoch 226
Validation auc = 0.690141
Epoch 227
Validation auc = 0.688732
Epoch 228
Validation auc = 0.691080
Epoch 229
Loss = 1.3906e-02, PNorm = 103.7600, GNorm = 0.4839, lr_0 = 6.8364e-04
Validation auc = 0.699531
Epoch 230
Validation auc = 0.707512
Epoch 231
Validation auc = 0.707042
Epoch 232
Validation auc = 0.706573
Epoch 233
Validation auc = 0.710798
Epoch 234
Loss = 7.1319e-03, PNorm = 103.8836, GNorm = 1.0186, lr_0 = 6.7779e-04
Validation auc = 0.710329
Epoch 235
Validation auc = 0.709859
Epoch 236
Validation auc = 0.712207
Epoch 237
Validation auc = 0.716901
Epoch 238
Validation auc = 0.720188
Epoch 239
Loss = 1.3616e-02, PNorm = 103.9867, GNorm = 0.1085, lr_0 = 6.7200e-04
Validation auc = 0.713615
Epoch 240
Validation auc = 0.706573
Epoch 241
Validation auc = 0.706103
Epoch 242
Validation auc = 0.704225
Epoch 243
Validation auc = 0.701408
Epoch 244
Loss = 1.0920e-01, PNorm = 104.1062, GNorm = 2.6104, lr_0 = 6.6625e-04
Validation auc = 0.704695
Epoch 245
Validation auc = 0.701878
Epoch 246
Validation auc = 0.701408
Epoch 247
Validation auc = 0.705164
Epoch 248
Validation auc = 0.700939
Epoch 249
Loss = 7.4403e-02, PNorm = 104.2503, GNorm = 0.4033, lr_0 = 6.6056e-04
Validation auc = 0.700469
Epoch 250
Validation auc = 0.696244
Epoch 251
Validation auc = 0.689202
Epoch 252
Validation auc = 0.688263
Epoch 253
Validation auc = 0.685446
Epoch 254
Loss = 1.0356e-02, PNorm = 104.3607, GNorm = 0.5587, lr_0 = 6.5491e-04
Validation auc = 0.684038
Epoch 255
Validation auc = 0.683568
Epoch 256
Validation auc = 0.686385
Epoch 257
Validation auc = 0.689202
Epoch 258
Validation auc = 0.697653
Epoch 259
Loss = 7.1290e-02, PNorm = 104.4653, GNorm = 0.7200, lr_0 = 6.4931e-04
Validation auc = 0.697653
Epoch 260
Validation auc = 0.695305
Epoch 261
Validation auc = 0.694836
Epoch 262
Validation auc = 0.692019
Epoch 263
Validation auc = 0.692958
Epoch 264
Loss = 1.0863e-01, PNorm = 104.5593, GNorm = 3.8211, lr_0 = 6.4376e-04
Validation auc = 0.692019
Epoch 265
Validation auc = 0.688263
Epoch 266
Validation auc = 0.686385
Epoch 267
Validation auc = 0.684507
Epoch 268
Validation auc = 0.687793
Epoch 269
Loss = 5.8951e-03, PNorm = 104.6651, GNorm = 0.8851, lr_0 = 6.3826e-04
Validation auc = 0.690141
Epoch 270
Validation auc = 0.692019
Epoch 271
Validation auc = 0.697183
Epoch 272
Validation auc = 0.702817
Epoch 273
Validation auc = 0.698122
Epoch 274
Loss = 4.3730e-02, PNorm = 104.7790, GNorm = 0.0742, lr_0 = 6.3280e-04
Validation auc = 0.697183
Epoch 275
Validation auc = 0.700469
Epoch 276
Validation auc = 0.706573
Epoch 277
Validation auc = 0.710329
Epoch 278
Validation auc = 0.708920
Epoch 279
Loss = 2.2755e-02, PNorm = 104.9135, GNorm = 1.3758, lr_0 = 6.2739e-04
Validation auc = 0.705634
Epoch 280
Validation auc = 0.711737
Epoch 281
Validation auc = 0.707512
Epoch 282
Validation auc = 0.698122
Epoch 283
Validation auc = 0.687324
Epoch 284
Loss = 1.3244e-02, PNorm = 105.0071, GNorm = 0.6110, lr_0 = 6.2203e-04
Validation auc = 0.686385
Epoch 285
Validation auc = 0.681690
Epoch 286
Validation auc = 0.684038
Epoch 287
Validation auc = 0.690141
Epoch 288
Validation auc = 0.702817
Epoch 289
Loss = 1.3611e-02, PNorm = 105.1617, GNorm = 0.7491, lr_0 = 6.1671e-04
Validation auc = 0.715493
Epoch 290
Validation auc = 0.717371
Epoch 291
Validation auc = 0.718310
Epoch 292
Validation auc = 0.713615
Epoch 293
Validation auc = 0.706573
Epoch 294
Loss = 1.5643e-02, PNorm = 105.3365, GNorm = 2.3659, lr_0 = 6.1144e-04
Validation auc = 0.711737
Epoch 295
Validation auc = 0.709859
Epoch 296
Validation auc = 0.710798
Epoch 297
Validation auc = 0.713146
Epoch 298
Validation auc = 0.714085
Epoch 299
Loss = 7.6978e-02, PNorm = 105.4667, GNorm = 2.9877, lr_0 = 6.0621e-04
Validation auc = 0.718310
Model 0 best validation auc = 0.749296 on epoch 103
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.713426
Ensemble test auc = 0.713426
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=2100, out_features=2100, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.2, inplace=False)
    (7): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 18,866,401
Moving model to cuda
Epoch 0
Validation auc = 0.714554
Epoch 1
Validation auc = 0.742254
Epoch 2
Validation auc = 0.711737
Epoch 3
Validation auc = 0.719249
Epoch 4
Loss = 1.0967e+00, PNorm = 94.3409, GNorm = 20.4743, lr_0 = 6.5000e-04
Validation auc = 0.736150
Epoch 5
Validation auc = 0.662911
Epoch 6
Validation auc = 0.696714
Epoch 7
Validation auc = 0.642723
Epoch 8
Validation auc = 0.602817
Epoch 9
Loss = 7.8532e-01, PNorm = 94.6612, GNorm = 19.3225, lr_0 = 9.9743e-04
Validation auc = 0.501408
Epoch 10
Validation auc = 0.579343
Epoch 11
Validation auc = 0.659624
Epoch 12
Validation auc = 0.698592
Epoch 13
Validation auc = 0.695305
Epoch 14
Loss = 1.0139e+00, PNorm = 94.9761, GNorm = 8.3114, lr_0 = 9.8890e-04
Validation auc = 0.664789
Epoch 15
Validation auc = 0.634742
Epoch 16
Validation auc = 0.681690
Epoch 17
Validation auc = 0.706573
Epoch 18
Validation auc = 0.715962
Epoch 19
Loss = 5.9636e-01, PNorm = 95.1905, GNorm = 7.1556, lr_0 = 9.8045e-04
Validation auc = 0.713615
Epoch 20
Validation auc = 0.701408
Epoch 21
Validation auc = 0.670892
Epoch 22
Validation auc = 0.691080
Epoch 23
Validation auc = 0.698122
Epoch 24
Loss = 4.3380e-01, PNorm = 95.3426, GNorm = 4.6192, lr_0 = 9.7207e-04
Validation auc = 0.699061
Epoch 25
Validation auc = 0.713615
Epoch 26
Validation auc = 0.707981
Epoch 27
Validation auc = 0.680751
Epoch 28
Validation auc = 0.646009
Epoch 29
Loss = 3.2076e-01, PNorm = 95.4780, GNorm = 2.9557, lr_0 = 9.6376e-04
Validation auc = 0.627700
Epoch 30
Validation auc = 0.641784
Epoch 31
Validation auc = 0.646948
Epoch 32
Validation auc = 0.651643
Epoch 33
Validation auc = 0.672300
Epoch 34
Loss = 2.6942e-01, PNorm = 95.6342, GNorm = 3.6345, lr_0 = 9.5552e-04
Validation auc = 0.693427
Epoch 35
Validation auc = 0.704225
Epoch 36
Validation auc = 0.687793
Epoch 37
Validation auc = 0.682160
Epoch 38
Validation auc = 0.686854
Epoch 39
Loss = 4.4019e-01, PNorm = 95.8278, GNorm = 5.2769, lr_0 = 9.4735e-04
Validation auc = 0.689202
Epoch 40
Validation auc = 0.690610
Epoch 41
Validation auc = 0.695775
Epoch 42
Validation auc = 0.689202
Epoch 43
Validation auc = 0.681221
Epoch 44
Loss = 2.0854e-01, PNorm = 96.0536, GNorm = 4.2062, lr_0 = 9.3925e-04
Validation auc = 0.683568
Epoch 45
Validation auc = 0.676056
Epoch 46
Validation auc = 0.689671
Epoch 47
Validation auc = 0.700469
Epoch 48
Validation auc = 0.689202
Epoch 49
Loss = 2.0169e-01, PNorm = 96.2728, GNorm = 3.7962, lr_0 = 9.3122e-04
Validation auc = 0.679812
Epoch 50
Validation auc = 0.694366
Epoch 51
Validation auc = 0.715023
Epoch 52
Validation auc = 0.715023
Epoch 53
Validation auc = 0.721127
Epoch 54
Loss = 2.6637e-01, PNorm = 96.5111, GNorm = 7.2435, lr_0 = 9.2326e-04
Validation auc = 0.710798
Epoch 55
Validation auc = 0.697653
Epoch 56
Validation auc = 0.708920
Epoch 57
Validation auc = 0.715962
Epoch 58
Validation auc = 0.727230
Epoch 59
Loss = 1.3164e-01, PNorm = 96.7460, GNorm = 1.9441, lr_0 = 9.1537e-04
Validation auc = 0.721127
Epoch 60
Validation auc = 0.725352
Epoch 61
Validation auc = 0.709859
Epoch 62
Validation auc = 0.667136
Epoch 63
Validation auc = 0.648357
Epoch 64
Loss = 1.9334e-01, PNorm = 97.0150, GNorm = 4.0514, lr_0 = 9.0754e-04
Validation auc = 0.661972
Epoch 65
Validation auc = 0.680282
Epoch 66
Validation auc = 0.679812
Epoch 67
Validation auc = 0.679343
Epoch 68
Validation auc = 0.669484
Epoch 69
Loss = 2.2961e-01, PNorm = 97.3036, GNorm = 3.5875, lr_0 = 8.9978e-04
Validation auc = 0.676995
Epoch 70
Validation auc = 0.679343
Epoch 71
Validation auc = 0.685915
Epoch 72
Validation auc = 0.706103
Epoch 73
Validation auc = 0.722535
Epoch 74
Loss = 3.5221e-01, PNorm = 97.5918, GNorm = 4.6825, lr_0 = 8.9209e-04
Validation auc = 0.726761
Epoch 75
Validation auc = 0.714554
Epoch 76
Validation auc = 0.695775
Epoch 77
Validation auc = 0.669953
Epoch 78
Validation auc = 0.653991
Epoch 79
Loss = 1.5898e-01, PNorm = 97.9188, GNorm = 2.8579, lr_0 = 8.8447e-04
Validation auc = 0.651174
Epoch 80
Validation auc = 0.657277
Epoch 81
Validation auc = 0.675117
Epoch 82
Validation auc = 0.683568
Epoch 83
Validation auc = 0.684977
Epoch 84
Loss = 8.2676e-02, PNorm = 98.2160, GNorm = 1.3333, lr_0 = 8.7691e-04
Validation auc = 0.692958
Epoch 85
Validation auc = 0.698122
Epoch 86
Validation auc = 0.704225
Epoch 87
Validation auc = 0.689202
Epoch 88
Validation auc = 0.683099
Epoch 89
Loss = 2.7700e-01, PNorm = 98.5110, GNorm = 4.3197, lr_0 = 8.6941e-04
Validation auc = 0.694366
Epoch 90
Validation auc = 0.715023
Epoch 91
Validation auc = 0.731925
Epoch 92
Validation auc = 0.746948
Epoch 93
Validation auc = 0.741315
Epoch 94
Loss = 1.8834e-01, PNorm = 98.7969, GNorm = 2.6690, lr_0 = 8.6198e-04
Validation auc = 0.733333
Epoch 95
Validation auc = 0.719249
Epoch 96
Validation auc = 0.721596
Epoch 97
Validation auc = 0.723944
Epoch 98
Validation auc = 0.727230
Epoch 99
Loss = 5.9995e-02, PNorm = 99.0725, GNorm = 1.9664, lr_0 = 8.5461e-04
Validation auc = 0.727230
Epoch 100
Validation auc = 0.738028
Epoch 101
Validation auc = 0.745070
Epoch 102
Validation auc = 0.742723
Epoch 103
Validation auc = 0.746009
Epoch 104
Loss = 1.5244e-01, PNorm = 99.3566, GNorm = 2.6682, lr_0 = 8.4730e-04
Validation auc = 0.753991
Epoch 105
Validation auc = 0.743192
Epoch 106
Validation auc = 0.738967
Epoch 107
Validation auc = 0.733333
Epoch 108
Validation auc = 0.721596
Epoch 109
Loss = 1.5031e-01, PNorm = 99.6371, GNorm = 2.9815, lr_0 = 8.4006e-04
Validation auc = 0.731455
Epoch 110
Validation auc = 0.742723
Epoch 111
Validation auc = 0.747887
Epoch 112
Validation auc = 0.737089
Epoch 113
Validation auc = 0.719249
Epoch 114
Loss = 1.1638e-01, PNorm = 99.9015, GNorm = 1.3162, lr_0 = 8.3288e-04
Validation auc = 0.720657
Epoch 115
Validation auc = 0.717371
Epoch 116
Validation auc = 0.715493
Epoch 117
Validation auc = 0.705164
Epoch 118
Validation auc = 0.707512
Epoch 119
Loss = 8.4481e-02, PNorm = 100.1550, GNorm = 3.1541, lr_0 = 8.2576e-04
Validation auc = 0.706573
Epoch 120
Validation auc = 0.691080
Epoch 121
Validation auc = 0.683568
Epoch 122
Validation auc = 0.691080
Epoch 123
Validation auc = 0.693897
Epoch 124
Loss = 1.7714e-01, PNorm = 100.3675, GNorm = 2.9338, lr_0 = 8.1870e-04
Validation auc = 0.697183
Epoch 125
Validation auc = 0.697653
Epoch 126
Validation auc = 0.702817
Epoch 127
Validation auc = 0.715493
Epoch 128
Validation auc = 0.718779
Epoch 129
Loss = 1.0714e-01, PNorm = 100.6205, GNorm = 2.7719, lr_0 = 8.1170e-04
Validation auc = 0.721596
Epoch 130
Validation auc = 0.722066
Epoch 131
Validation auc = 0.721596
Epoch 132
Validation auc = 0.711268
Epoch 133
Validation auc = 0.694366
Epoch 134
Loss = 1.0708e-01, PNorm = 100.8937, GNorm = 2.6373, lr_0 = 8.0476e-04
Validation auc = 0.668075
Epoch 135
Validation auc = 0.660094
Epoch 136
Validation auc = 0.650235
Epoch 137
Validation auc = 0.644601
Epoch 138
Validation auc = 0.657746
Epoch 139
Loss = 5.0255e-02, PNorm = 101.1394, GNorm = 0.8451, lr_0 = 7.9788e-04
Validation auc = 0.677465
Epoch 140
Validation auc = 0.707042
Epoch 141
Validation auc = 0.722535
Epoch 142
Validation auc = 0.731925
Epoch 143
Validation auc = 0.733333
Epoch 144
Loss = 8.7002e-02, PNorm = 101.3730, GNorm = 1.4494, lr_0 = 7.9106e-04
Validation auc = 0.738028
Epoch 145
Validation auc = 0.738498
Epoch 146
Validation auc = 0.734742
Epoch 147
Validation auc = 0.733803
Epoch 148
Validation auc = 0.738028
Epoch 149
Loss = 1.5834e-01, PNorm = 101.5854, GNorm = 3.6126, lr_0 = 7.8430e-04
Validation auc = 0.726291
Epoch 150
Validation auc = 0.713615
Epoch 151
Validation auc = 0.708451
Epoch 152
Validation auc = 0.709390
Epoch 153
Validation auc = 0.718310
Epoch 154
Loss = 5.4100e-02, PNorm = 101.7631, GNorm = 0.4389, lr_0 = 7.7759e-04
Validation auc = 0.716901
Epoch 155
Validation auc = 0.714554
Epoch 156
Validation auc = 0.707042
Epoch 157
Validation auc = 0.701878
Epoch 158
Validation auc = 0.700469
Epoch 159
Loss = 1.0466e-01, PNorm = 101.9342, GNorm = 0.5251, lr_0 = 7.7095e-04
Validation auc = 0.702817
Epoch 160
Validation auc = 0.706573
Epoch 161
Validation auc = 0.708451
Epoch 162
Validation auc = 0.713146
Epoch 163
Validation auc = 0.716432
Epoch 164
Loss = 1.0057e-01, PNorm = 102.1054, GNorm = 2.1499, lr_0 = 7.6436e-04
Validation auc = 0.720657
Epoch 165
Validation auc = 0.723005
Epoch 166
Validation auc = 0.713615
Epoch 167
Validation auc = 0.703286
Epoch 168
Validation auc = 0.692488
Epoch 169
Loss = 5.7805e-02, PNorm = 102.3356, GNorm = 1.3421, lr_0 = 7.5782e-04
Validation auc = 0.689671
Epoch 170
Validation auc = 0.688732
Epoch 171
Validation auc = 0.696714
Epoch 172
Validation auc = 0.695305
Epoch 173
Validation auc = 0.709390
Epoch 174
Loss = 9.6142e-02, PNorm = 102.5504, GNorm = 2.0145, lr_0 = 7.5134e-04
Validation auc = 0.707981
Epoch 175
Validation auc = 0.704225
Epoch 176
Validation auc = 0.704695
Epoch 177
Validation auc = 0.701408
Epoch 178
Validation auc = 0.692958
Epoch 179
Loss = 2.0791e-01, PNorm = 102.8147, GNorm = 3.4463, lr_0 = 7.4492e-04
Validation auc = 0.704225
Epoch 180
Validation auc = 0.704225
Epoch 181
Validation auc = 0.706573
Epoch 182
Validation auc = 0.713146
Epoch 183
Validation auc = 0.705634
Epoch 184
Loss = 1.1385e-01, PNorm = 103.0265, GNorm = 3.4323, lr_0 = 7.3855e-04
Validation auc = 0.704695
Epoch 185
Validation auc = 0.703756
Epoch 186
Validation auc = 0.701878
Epoch 187
Validation auc = 0.702817
Epoch 188
Validation auc = 0.709390
Epoch 189
Loss = 1.3887e-01, PNorm = 103.2354, GNorm = 4.9719, lr_0 = 7.3224e-04
Validation auc = 0.714554
Epoch 190
Validation auc = 0.710329
Epoch 191
Validation auc = 0.704225
Epoch 192
Validation auc = 0.701408
Epoch 193
Validation auc = 0.698592
Epoch 194
Loss = 6.6337e-02, PNorm = 103.4323, GNorm = 1.1823, lr_0 = 7.2598e-04
Validation auc = 0.711737
Epoch 195
Validation auc = 0.713615
Epoch 196
Validation auc = 0.723944
Epoch 197
Validation auc = 0.729108
Epoch 198
Validation auc = 0.729577
Epoch 199
Loss = 3.0595e-02, PNorm = 103.6168, GNorm = 2.8276, lr_0 = 7.1977e-04
Validation auc = 0.734742
Epoch 200
Validation auc = 0.726761
Epoch 201
Validation auc = 0.721127
Epoch 202
Validation auc = 0.716901
Epoch 203
Validation auc = 0.708920
Epoch 204
Loss = 2.3444e-02, PNorm = 103.7589, GNorm = 1.5468, lr_0 = 7.1362e-04
Validation auc = 0.700469
Epoch 205
Validation auc = 0.697183
Epoch 206
Validation auc = 0.699531
Epoch 207
Validation auc = 0.703756
Epoch 208
Validation auc = 0.711268
Epoch 209
Loss = 3.8171e-02, PNorm = 103.8900, GNorm = 1.3836, lr_0 = 7.0752e-04
Validation auc = 0.713615
Epoch 210
Validation auc = 0.721127
Epoch 211
Validation auc = 0.720657
Epoch 212
Validation auc = 0.712207
Epoch 213
Validation auc = 0.714085
Epoch 214
Loss = 6.3826e-03, PNorm = 104.0024, GNorm = 0.1682, lr_0 = 7.0147e-04
Validation auc = 0.716901
Epoch 215
Validation auc = 0.711737
Epoch 216
Validation auc = 0.704695
Epoch 217
Validation auc = 0.709859
Epoch 218
Validation auc = 0.712676
Epoch 219
Loss = 2.2313e-02, PNorm = 104.1090, GNorm = 0.0705, lr_0 = 6.9548e-04
Validation auc = 0.714554
Epoch 220
Validation auc = 0.710329
Epoch 221
Validation auc = 0.701408
Epoch 222
Validation auc = 0.701408
Epoch 223
Validation auc = 0.702817
Epoch 224
Loss = 1.2350e-01, PNorm = 104.2361, GNorm = 0.8190, lr_0 = 6.8953e-04
Validation auc = 0.705164
Epoch 225
Validation auc = 0.704225
Epoch 226
Validation auc = 0.709390
Epoch 227
Validation auc = 0.712676
Epoch 228
Validation auc = 0.715023
Epoch 229
Loss = 2.1259e-02, PNorm = 104.3712, GNorm = 0.3295, lr_0 = 6.8364e-04
Validation auc = 0.708451
Epoch 230
Validation auc = 0.701878
Epoch 231
Validation auc = 0.700000
Epoch 232
Validation auc = 0.707512
Epoch 233
Validation auc = 0.715493
Epoch 234
Loss = 2.6127e-02, PNorm = 104.5102, GNorm = 1.1006, lr_0 = 6.7779e-04
Validation auc = 0.718779
Epoch 235
Validation auc = 0.718310
Epoch 236
Validation auc = 0.710798
Epoch 237
Validation auc = 0.708451
Epoch 238
Validation auc = 0.709390
Epoch 239
Loss = 1.4964e-02, PNorm = 104.6553, GNorm = 0.4519, lr_0 = 6.7200e-04
Validation auc = 0.711737
Epoch 240
Validation auc = 0.715962
Epoch 241
Validation auc = 0.722535
Epoch 242
Validation auc = 0.723944
Epoch 243
Validation auc = 0.727230
Epoch 244
Loss = 2.3509e-02, PNorm = 104.7980, GNorm = 1.6579, lr_0 = 6.6625e-04
Validation auc = 0.730047
Epoch 245
Validation auc = 0.729577
Epoch 246
Validation auc = 0.729577
Epoch 247
Validation auc = 0.722066
Epoch 248
Validation auc = 0.723944
Epoch 249
Loss = 1.0794e-01, PNorm = 104.9440, GNorm = 0.5276, lr_0 = 6.6056e-04
Validation auc = 0.720657
Epoch 250
Validation auc = 0.715023
Epoch 251
Validation auc = 0.706573
Epoch 252
Validation auc = 0.704225
Epoch 253
Validation auc = 0.705164
Epoch 254
Loss = 6.3432e-03, PNorm = 105.0658, GNorm = 0.1542, lr_0 = 6.5491e-04
Validation auc = 0.706103
Epoch 255
Validation auc = 0.700939
Epoch 256
Validation auc = 0.699531
Epoch 257
Validation auc = 0.699061
Epoch 258
Validation auc = 0.700000
Epoch 259
Loss = 2.0709e-02, PNorm = 105.1688, GNorm = 1.4784, lr_0 = 6.4931e-04
Validation auc = 0.704225
Epoch 260
Validation auc = 0.712207
Epoch 261
Validation auc = 0.711737
Epoch 262
Validation auc = 0.713615
Epoch 263
Validation auc = 0.712207
Epoch 264
Loss = 1.8106e-02, PNorm = 105.2762, GNorm = 0.8170, lr_0 = 6.4376e-04
Validation auc = 0.710329
Epoch 265
Validation auc = 0.711737
Epoch 266
Validation auc = 0.707512
Epoch 267
Validation auc = 0.704225
Epoch 268
Validation auc = 0.699061
Epoch 269
Loss = 1.7043e-02, PNorm = 105.3639, GNorm = 0.9611, lr_0 = 6.3826e-04
Validation auc = 0.697183
Epoch 270
Validation auc = 0.693427
Epoch 271
Validation auc = 0.695775
Epoch 272
Validation auc = 0.699531
Epoch 273
Validation auc = 0.701878
Epoch 274
Loss = 8.7915e-03, PNorm = 105.4576, GNorm = 0.1259, lr_0 = 6.3280e-04
Validation auc = 0.706573
Epoch 275
Validation auc = 0.706103
Epoch 276
Validation auc = 0.712676
Epoch 277
Validation auc = 0.713146
Epoch 278
Validation auc = 0.714554
Epoch 279
Loss = 6.5932e-03, PNorm = 105.5709, GNorm = 0.4460, lr_0 = 6.2739e-04
Validation auc = 0.716901
Epoch 280
Validation auc = 0.717840
Epoch 281
Validation auc = 0.714554
Epoch 282
Validation auc = 0.711268
Epoch 283
Validation auc = 0.718310
Epoch 284
Loss = 1.0549e-02, PNorm = 105.6540, GNorm = 0.0278, lr_0 = 6.2203e-04
Validation auc = 0.721127
Epoch 285
Validation auc = 0.723005
Epoch 286
Validation auc = 0.720188
Epoch 287
Validation auc = 0.716432
Epoch 288
Validation auc = 0.719249
Epoch 289
Loss = 1.8526e-02, PNorm = 105.7368, GNorm = 1.2411, lr_0 = 6.1671e-04
Validation auc = 0.718310
Epoch 290
Validation auc = 0.720657
Epoch 291
Validation auc = 0.719718
Epoch 292
Validation auc = 0.710798
Epoch 293
Validation auc = 0.699061
Epoch 294
Loss = 3.1921e-02, PNorm = 105.8381, GNorm = 2.6917, lr_0 = 6.1144e-04
Validation auc = 0.694836
Epoch 295
Validation auc = 0.696244
Epoch 296
Validation auc = 0.705164
Epoch 297
Validation auc = 0.711737
Epoch 298
Validation auc = 0.713615
Epoch 299
Loss = 1.0375e-01, PNorm = 105.9728, GNorm = 2.6668, lr_0 = 6.0621e-04
Validation auc = 0.710798
Model 0 best validation auc = 0.753991 on epoch 104
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.815172
Ensemble test auc = 0.815172
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=2100, out_features=2100, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.2, inplace=False)
    (7): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 18,866,401
Moving model to cuda
Epoch 0
Validation auc = 0.760563
Epoch 1
Validation auc = 0.716901
Epoch 2
Validation auc = 0.735211
Epoch 3
Validation auc = 0.670423
Epoch 4
Loss = 1.3797e+00, PNorm = 94.3474, GNorm = 24.5123, lr_0 = 6.5000e-04
Validation auc = 0.670423
Epoch 5
Validation auc = 0.670423
Epoch 6
Validation auc = 0.649296
Epoch 7
Validation auc = 0.698592
Epoch 8
Validation auc = 0.666667
Epoch 9
Loss = 8.3885e-01, PNorm = 94.6853, GNorm = 13.2871, lr_0 = 9.9743e-04
Validation auc = 0.708451
Epoch 10
Validation auc = 0.711268
Epoch 11
Validation auc = 0.690610
Epoch 12
Validation auc = 0.675587
Epoch 13
Validation auc = 0.723474
Epoch 14
Loss = 4.3465e-01, PNorm = 95.0262, GNorm = 6.7915, lr_0 = 9.8890e-04
Validation auc = 0.726761
Epoch 15
Validation auc = 0.684977
Epoch 16
Validation auc = 0.628638
Epoch 17
Validation auc = 0.669484
Epoch 18
Validation auc = 0.690141
Epoch 19
Loss = 3.8378e-01, PNorm = 95.2450, GNorm = 4.4897, lr_0 = 9.8045e-04
Validation auc = 0.698592
Epoch 20
Validation auc = 0.677465
Epoch 21
Validation auc = 0.615962
Epoch 22
Validation auc = 0.588732
Epoch 23
Validation auc = 0.550235
Epoch 24
Loss = 4.6852e-01, PNorm = 95.4070, GNorm = 10.1345, lr_0 = 9.7207e-04
Validation auc = 0.649765
Epoch 25
Validation auc = 0.676056
Epoch 26
Validation auc = 0.673709
Epoch 27
Validation auc = 0.664319
Epoch 28
Validation auc = 0.608920
Epoch 29
Loss = 5.8469e-01, PNorm = 95.5618, GNorm = 9.9092, lr_0 = 9.6376e-04
Validation auc = 0.608451
Epoch 30
Validation auc = 0.630986
Epoch 31
Validation auc = 0.671831
Epoch 32
Validation auc = 0.714085
Epoch 33
Validation auc = 0.719249
Epoch 34
Loss = 2.7642e-01, PNorm = 95.7308, GNorm = 5.9815, lr_0 = 9.5552e-04
Validation auc = 0.702347
Epoch 35
Validation auc = 0.689671
Epoch 36
Validation auc = 0.669484
Epoch 37
Validation auc = 0.653521
Epoch 38
Validation auc = 0.651174
Epoch 39
Loss = 2.8815e-01, PNorm = 95.9468, GNorm = 4.1282, lr_0 = 9.4735e-04
Validation auc = 0.662911
Epoch 40
Validation auc = 0.642254
Epoch 41
Validation auc = 0.647418
Epoch 42
Validation auc = 0.677465
Epoch 43
Validation auc = 0.683099
Epoch 44
Loss = 3.7433e-01, PNorm = 96.1887, GNorm = 5.2124, lr_0 = 9.3925e-04
Validation auc = 0.681690
Epoch 45
Validation auc = 0.717840
Epoch 46
Validation auc = 0.730047
Epoch 47
Validation auc = 0.723944
Epoch 48
Validation auc = 0.730516
Epoch 49
Loss = 2.6136e-01, PNorm = 96.4694, GNorm = 2.8267, lr_0 = 9.3122e-04
Validation auc = 0.716901
Epoch 50
Validation auc = 0.697183
Epoch 51
Validation auc = 0.698592
Epoch 52
Validation auc = 0.704225
Epoch 53
Validation auc = 0.712676
Epoch 54
Loss = 2.1549e-01, PNorm = 96.7148, GNorm = 3.8145, lr_0 = 9.2326e-04
Validation auc = 0.730986
Epoch 55
Validation auc = 0.717371
Epoch 56
Validation auc = 0.718779
Epoch 57
Validation auc = 0.705634
Epoch 58
Validation auc = 0.684977
Epoch 59
Loss = 2.4329e-01, PNorm = 96.9471, GNorm = 2.7041, lr_0 = 9.1537e-04
Validation auc = 0.667606
Epoch 60
Validation auc = 0.692019
Epoch 61
Validation auc = 0.710798
Epoch 62
Validation auc = 0.715962
Epoch 63
Validation auc = 0.716432
Epoch 64
Loss = 2.2125e-01, PNorm = 97.2023, GNorm = 3.5193, lr_0 = 9.0754e-04
Validation auc = 0.730047
Epoch 65
Validation auc = 0.723474
Epoch 66
Validation auc = 0.717371
Epoch 67
Validation auc = 0.736150
Epoch 68
Validation auc = 0.746948
Epoch 69
Loss = 1.8142e-01, PNorm = 97.4791, GNorm = 4.8811, lr_0 = 8.9978e-04
Validation auc = 0.745540
Epoch 70
Validation auc = 0.733803
Epoch 71
Validation auc = 0.713146
Epoch 72
Validation auc = 0.690610
Epoch 73
Validation auc = 0.686854
Epoch 74
Loss = 1.5163e-01, PNorm = 97.7609, GNorm = 1.6664, lr_0 = 8.9209e-04
Validation auc = 0.702817
Epoch 75
Validation auc = 0.718310
Epoch 76
Validation auc = 0.716432
Epoch 77
Validation auc = 0.699531
Epoch 78
Validation auc = 0.670892
Epoch 79
Loss = 3.4989e-01, PNorm = 98.0411, GNorm = 6.0015, lr_0 = 8.8447e-04
Validation auc = 0.638498
Epoch 80
Validation auc = 0.653521
Epoch 81
Validation auc = 0.692019
Epoch 82
Validation auc = 0.695305
Epoch 83
Validation auc = 0.695305
Epoch 84
Loss = 2.8239e-01, PNorm = 98.2945, GNorm = 4.7808, lr_0 = 8.7691e-04
Validation auc = 0.685446
Epoch 85
Validation auc = 0.694836
Epoch 86
Validation auc = 0.714554
Epoch 87
Validation auc = 0.715962
Epoch 88
Validation auc = 0.725352
Epoch 89
Loss = 2.2204e-01, PNorm = 98.5859, GNorm = 2.7806, lr_0 = 8.6941e-04
Validation auc = 0.720188
Epoch 90
Validation auc = 0.718779
Epoch 91
Validation auc = 0.715493
Epoch 92
Validation auc = 0.703286
Epoch 93
Validation auc = 0.678873
Epoch 94
Loss = 1.1869e-01, PNorm = 98.9303, GNorm = 2.7763, lr_0 = 8.6198e-04
Validation auc = 0.660563
Epoch 95
Validation auc = 0.684038
Epoch 96
Validation auc = 0.709390
Epoch 97
Validation auc = 0.716901
Epoch 98
Validation auc = 0.723474
Epoch 99
Loss = 1.1830e-01, PNorm = 99.2266, GNorm = 4.3624, lr_0 = 8.5461e-04
Validation auc = 0.721596
Epoch 100
Validation auc = 0.707981
Epoch 101
Validation auc = 0.698122
Epoch 102
Validation auc = 0.691080
Epoch 103
Validation auc = 0.685915
Epoch 104
Loss = 1.3698e-01, PNorm = 99.4711, GNorm = 2.9576, lr_0 = 8.4730e-04
Validation auc = 0.686854
Epoch 105
Validation auc = 0.701878
Epoch 106
Validation auc = 0.715962
Epoch 107
Validation auc = 0.724883
Epoch 108
Validation auc = 0.724413
Epoch 109
Loss = 9.7367e-02, PNorm = 99.7068, GNorm = 3.4389, lr_0 = 8.4006e-04
Validation auc = 0.728169
Epoch 110
Validation auc = 0.726761
Epoch 111
Validation auc = 0.722066
Epoch 112
Validation auc = 0.708920
Epoch 113
Validation auc = 0.709390
Epoch 114
Loss = 9.0920e-02, PNorm = 99.9781, GNorm = 5.6274, lr_0 = 8.3288e-04
Validation auc = 0.702347
Epoch 115
Validation auc = 0.701878
Epoch 116
Validation auc = 0.701878
Epoch 117
Validation auc = 0.705164
Epoch 118
Validation auc = 0.715023
Epoch 119
Loss = 4.9318e-02, PNorm = 100.2223, GNorm = 0.7810, lr_0 = 8.2576e-04
Validation auc = 0.713615
Epoch 120
Validation auc = 0.712207
Epoch 121
Validation auc = 0.709390
Epoch 122
Validation auc = 0.721127
Epoch 123
Validation auc = 0.747418
Epoch 124
Loss = 1.4145e-01, PNorm = 100.4402, GNorm = 1.1759, lr_0 = 8.1870e-04
Validation auc = 0.753991
Epoch 125
Validation auc = 0.751174
Epoch 126
Validation auc = 0.752582
Epoch 127
Validation auc = 0.753052
Epoch 128
Validation auc = 0.755869
Epoch 129
Loss = 4.2545e-02, PNorm = 100.6550, GNorm = 0.4057, lr_0 = 8.1170e-04
Validation auc = 0.760094
Epoch 130
Validation auc = 0.753991
Epoch 131
Validation auc = 0.745540
Epoch 132
Validation auc = 0.730516
Epoch 133
Validation auc = 0.726291
Epoch 134
Loss = 1.6582e-02, PNorm = 100.8546, GNorm = 0.8716, lr_0 = 8.0476e-04
Validation auc = 0.724883
Epoch 135
Validation auc = 0.719249
Epoch 136
Validation auc = 0.721596
Epoch 137
Validation auc = 0.736150
Epoch 138
Validation auc = 0.741315
Epoch 139
Loss = 1.6905e-01, PNorm = 101.0646, GNorm = 3.9097, lr_0 = 7.9788e-04
Validation auc = 0.748826
Epoch 140
Validation auc = 0.753521
Epoch 141
Validation auc = 0.743662
Epoch 142
Validation auc = 0.746948
Epoch 143
Validation auc = 0.749296
Epoch 144
Loss = 2.0331e-01, PNorm = 101.3372, GNorm = 1.9539, lr_0 = 7.9106e-04
Validation auc = 0.739437
Epoch 145
Validation auc = 0.743192
Epoch 146
Validation auc = 0.746009
Epoch 147
Validation auc = 0.746948
Epoch 148
Validation auc = 0.750704
Epoch 149
Loss = 9.1226e-02, PNorm = 101.6319, GNorm = 1.2358, lr_0 = 7.8430e-04
Validation auc = 0.750704
Epoch 150
Validation auc = 0.743192
Epoch 151
Validation auc = 0.738967
Epoch 152
Validation auc = 0.725352
Epoch 153
Validation auc = 0.722535
Epoch 154
Loss = 6.6629e-02, PNorm = 101.9010, GNorm = 0.6727, lr_0 = 7.7759e-04
Validation auc = 0.704695
Epoch 155
Validation auc = 0.692958
Epoch 156
Validation auc = 0.694366
Epoch 157
Validation auc = 0.710798
Epoch 158
Validation auc = 0.721596
Epoch 159
Loss = 3.1400e-02, PNorm = 102.1109, GNorm = 0.7838, lr_0 = 7.7095e-04
Validation auc = 0.726291
Epoch 160
Validation auc = 0.724883
Epoch 161
Validation auc = 0.727700
Epoch 162
Validation auc = 0.735211
Epoch 163
Validation auc = 0.738498
Epoch 164
Loss = 4.9644e-02, PNorm = 102.2847, GNorm = 1.4443, lr_0 = 7.6436e-04
Validation auc = 0.736150
Epoch 165
Validation auc = 0.738498
Epoch 166
Validation auc = 0.736620
Epoch 167
Validation auc = 0.733803
Epoch 168
Validation auc = 0.736620
Epoch 169
Loss = 1.0441e-02, PNorm = 102.4625, GNorm = 0.2986, lr_0 = 7.5782e-04
Validation auc = 0.732864
Epoch 170
Validation auc = 0.731925
Epoch 171
Validation auc = 0.732394
Epoch 172
Validation auc = 0.729108
Epoch 173
Validation auc = 0.725822
Epoch 174
Loss = 8.8489e-02, PNorm = 102.6144, GNorm = 0.2680, lr_0 = 7.5134e-04
Validation auc = 0.722066
Epoch 175
Validation auc = 0.718779
Epoch 176
Validation auc = 0.720188
Epoch 177
Validation auc = 0.713146
Epoch 178
Validation auc = 0.705634
Epoch 179
Loss = 8.2433e-02, PNorm = 102.7697, GNorm = 0.3748, lr_0 = 7.4492e-04
Validation auc = 0.699531
Epoch 180
Validation auc = 0.697183
Epoch 181
Validation auc = 0.695775
Epoch 182
Validation auc = 0.694836
Epoch 183
Validation auc = 0.711268
Epoch 184
Loss = 1.3986e-01, PNorm = 102.9720, GNorm = 1.8840, lr_0 = 7.3855e-04
Validation auc = 0.715962
Epoch 185
Validation auc = 0.722066
Epoch 186
Validation auc = 0.715493
Epoch 187
Validation auc = 0.679812
Epoch 188
Validation auc = 0.666667
Epoch 189
Loss = 4.7175e-02, PNorm = 103.1837, GNorm = 1.0822, lr_0 = 7.3224e-04
Validation auc = 0.659624
Epoch 190
Validation auc = 0.642723
Epoch 191
Validation auc = 0.644601
Epoch 192
Validation auc = 0.648826
Epoch 193
Validation auc = 0.655869
Epoch 194
Loss = 8.5740e-02, PNorm = 103.4160, GNorm = 2.1995, lr_0 = 7.2598e-04
Validation auc = 0.655399
Epoch 195
Validation auc = 0.661502
Epoch 196
Validation auc = 0.665728
Epoch 197
Validation auc = 0.660563
Epoch 198
Validation auc = 0.666667
Epoch 199
Loss = 9.4963e-02, PNorm = 103.6594, GNorm = 2.6755, lr_0 = 7.1977e-04
Validation auc = 0.677465
Epoch 200
Validation auc = 0.684977
Epoch 201
Validation auc = 0.698122
Epoch 202
Validation auc = 0.709859
Epoch 203
Validation auc = 0.714085
Epoch 204
Loss = 3.5914e-02, PNorm = 103.8706, GNorm = 2.4842, lr_0 = 7.1362e-04
Validation auc = 0.713146
Epoch 205
Validation auc = 0.713615
Epoch 206
Validation auc = 0.708920
Epoch 207
Validation auc = 0.702817
Epoch 208
Validation auc = 0.697653
Epoch 209
Loss = 3.6358e-02, PNorm = 104.0346, GNorm = 1.0170, lr_0 = 7.0752e-04
Validation auc = 0.692019
Epoch 210
Validation auc = 0.691549
Epoch 211
Validation auc = 0.691549
Epoch 212
Validation auc = 0.708451
Epoch 213
Validation auc = 0.720188
Epoch 214
Loss = 1.2292e-01, PNorm = 104.1704, GNorm = 5.1915, lr_0 = 7.0147e-04
Validation auc = 0.714085
Epoch 215
Validation auc = 0.705164
Epoch 216
Validation auc = 0.704695
Epoch 217
Validation auc = 0.696714
Epoch 218
Validation auc = 0.689671
Epoch 219
Loss = 1.6914e-02, PNorm = 104.2794, GNorm = 0.5895, lr_0 = 6.9548e-04
Validation auc = 0.679812
Epoch 220
Validation auc = 0.681690
Epoch 221
Validation auc = 0.679812
Epoch 222
Validation auc = 0.680282
Epoch 223
Validation auc = 0.687324
Epoch 224
Loss = 1.0117e-02, PNorm = 104.4231, GNorm = 0.5648, lr_0 = 6.8953e-04
Validation auc = 0.689202
Epoch 225
Validation auc = 0.693427
Epoch 226
Validation auc = 0.692019
Epoch 227
Validation auc = 0.698592
Epoch 228
Validation auc = 0.706103
Epoch 229
Loss = 2.1153e-02, PNorm = 104.5969, GNorm = 1.5363, lr_0 = 6.8364e-04
Validation auc = 0.705634
Epoch 230
Validation auc = 0.706103
Epoch 231
Validation auc = 0.705634
Epoch 232
Validation auc = 0.696244
Epoch 233
Validation auc = 0.694366
Epoch 234
Loss = 7.5305e-02, PNorm = 104.7900, GNorm = 5.4171, lr_0 = 6.7779e-04
Validation auc = 0.692488
Epoch 235
Validation auc = 0.682629
Epoch 236
Validation auc = 0.680751
Epoch 237
Validation auc = 0.680282
Epoch 238
Validation auc = 0.684507
Epoch 239
Loss = 1.3975e-01, PNorm = 104.9567, GNorm = 3.0243, lr_0 = 6.7200e-04
Validation auc = 0.689202
Epoch 240
Validation auc = 0.687793
Epoch 241
Validation auc = 0.685446
Epoch 242
Validation auc = 0.685446
Epoch 243
Validation auc = 0.680751
Epoch 244
Loss = 1.0410e-01, PNorm = 105.1563, GNorm = 1.7983, lr_0 = 6.6625e-04
Validation auc = 0.686385
Epoch 245
Validation auc = 0.688732
Epoch 246
Validation auc = 0.692958
Epoch 247
Validation auc = 0.696244
Epoch 248
Validation auc = 0.687324
Epoch 249
Loss = 2.2832e-02, PNorm = 105.3330, GNorm = 1.0444, lr_0 = 6.6056e-04
Validation auc = 0.681221
Epoch 250
Validation auc = 0.678404
Epoch 251
Validation auc = 0.675587
Epoch 252
Validation auc = 0.672770
Epoch 253
Validation auc = 0.668075
Epoch 254
Loss = 2.5315e-02, PNorm = 105.4928, GNorm = 1.5076, lr_0 = 6.5491e-04
Validation auc = 0.665728
Epoch 255
Validation auc = 0.667606
Epoch 256
Validation auc = 0.670423
Epoch 257
Validation auc = 0.673709
Epoch 258
Validation auc = 0.677934
Epoch 259
Loss = 5.4229e-02, PNorm = 105.5947, GNorm = 1.7062, lr_0 = 6.4931e-04
Validation auc = 0.683099
Epoch 260
Validation auc = 0.687793
Epoch 261
Validation auc = 0.689202
Epoch 262
Validation auc = 0.683568
Epoch 263
Validation auc = 0.684038
Epoch 264
Loss = 2.5980e-02, PNorm = 105.7222, GNorm = 1.1390, lr_0 = 6.4376e-04
Validation auc = 0.677934
Epoch 265
Validation auc = 0.675117
Epoch 266
Validation auc = 0.675117
Epoch 267
Validation auc = 0.673239
Epoch 268
Validation auc = 0.675117
Epoch 269
Loss = 4.6893e-02, PNorm = 105.8544, GNorm = 1.2242, lr_0 = 6.3826e-04
Validation auc = 0.672770
Epoch 270
Validation auc = 0.679343
Epoch 271
Validation auc = 0.683099
Epoch 272
Validation auc = 0.686854
Epoch 273
Validation auc = 0.684507
Epoch 274
Loss = 1.9451e-02, PNorm = 105.9584, GNorm = 0.1699, lr_0 = 6.3280e-04
Validation auc = 0.683099
Epoch 275
Validation auc = 0.683568
Epoch 276
Validation auc = 0.680282
Epoch 277
Validation auc = 0.682160
Epoch 278
Validation auc = 0.681690
Epoch 279
Loss = 4.3803e-02, PNorm = 106.0609, GNorm = 2.0729, lr_0 = 6.2739e-04
Validation auc = 0.680751
Epoch 280
Validation auc = 0.680282
Epoch 281
Validation auc = 0.682629
Epoch 282
Validation auc = 0.684038
Epoch 283
Validation auc = 0.680282
Epoch 284
Loss = 6.0454e-02, PNorm = 106.1869, GNorm = 1.6238, lr_0 = 6.2203e-04
Validation auc = 0.679343
Epoch 285
Validation auc = 0.681690
Epoch 286
Validation auc = 0.686385
Epoch 287
Validation auc = 0.689202
Epoch 288
Validation auc = 0.690141
Epoch 289
Loss = 1.4853e-02, PNorm = 106.2826, GNorm = 0.3758, lr_0 = 6.1671e-04
Validation auc = 0.694366
Epoch 290
Validation auc = 0.695775
Epoch 291
Validation auc = 0.697653
Epoch 292
Validation auc = 0.697183
Epoch 293
Validation auc = 0.694836
Epoch 294
Loss = 2.2600e-02, PNorm = 106.3582, GNorm = 0.1183, lr_0 = 6.1144e-04
Validation auc = 0.687324
Epoch 295
Validation auc = 0.683568
Epoch 296
Validation auc = 0.682629
Epoch 297
Validation auc = 0.680282
Epoch 298
Validation auc = 0.676526
Epoch 299
Loss = 7.9291e-03, PNorm = 106.4193, GNorm = 0.1268, lr_0 = 6.0621e-04
Validation auc = 0.675587
Model 0 best validation auc = 0.760563 on epoch 0
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.829621
Ensemble test auc = 0.829621
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=2100, out_features=2100, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.2, inplace=False)
    (7): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 18,866,401
Moving model to cuda
Epoch 0
Validation auc = 0.723944
Epoch 1
Validation auc = 0.696714
Epoch 2
Validation auc = 0.722535
Epoch 3
Validation auc = 0.710329
Epoch 4
Loss = 9.6477e-01, PNorm = 94.3451, GNorm = 11.6420, lr_0 = 6.5000e-04
Validation auc = 0.680282
Epoch 5
Validation auc = 0.636620
Epoch 6
Validation auc = 0.708920
Epoch 7
Validation auc = 0.681690
Epoch 8
Validation auc = 0.693662
Epoch 9
Loss = 1.3205e+00, PNorm = 94.6721, GNorm = 22.6667, lr_0 = 9.9743e-04
Validation auc = 0.697653
Epoch 10
Validation auc = 0.682629
Epoch 11
Validation auc = 0.655399
Epoch 12
Validation auc = 0.625352
Epoch 13
Validation auc = 0.705634
Epoch 14
Loss = 7.3732e-01, PNorm = 94.9999, GNorm = 11.5588, lr_0 = 9.8890e-04
Validation auc = 0.682629
Epoch 15
Validation auc = 0.686854
Epoch 16
Validation auc = 0.695305
Epoch 17
Validation auc = 0.699531
Epoch 18
Validation auc = 0.697183
Epoch 19
Loss = 4.5446e-01, PNorm = 95.2182, GNorm = 8.0857, lr_0 = 9.8045e-04
Validation auc = 0.698122
Epoch 20
Validation auc = 0.721127
Epoch 21
Validation auc = 0.714554
Epoch 22
Validation auc = 0.693897
Epoch 23
Validation auc = 0.661502
Epoch 24
Loss = 4.3970e-01, PNorm = 95.3756, GNorm = 10.0415, lr_0 = 9.7207e-04
Validation auc = 0.647887
Epoch 25
Validation auc = 0.637089
Epoch 26
Validation auc = 0.653521
Epoch 27
Validation auc = 0.697653
Epoch 28
Validation auc = 0.712676
Epoch 29
Loss = 4.9334e-01, PNorm = 95.5122, GNorm = 7.2095, lr_0 = 9.6376e-04
Validation auc = 0.696244
Epoch 30
Validation auc = 0.702817
Epoch 31
Validation auc = 0.722535
Epoch 32
Validation auc = 0.706573
Epoch 33
Validation auc = 0.686385
Epoch 34
Loss = 3.2755e-01, PNorm = 95.6707, GNorm = 6.6624, lr_0 = 9.5552e-04
Validation auc = 0.655869
Epoch 35
Validation auc = 0.643662
Epoch 36
Validation auc = 0.658685
Epoch 37
Validation auc = 0.673709
Epoch 38
Validation auc = 0.679812
Epoch 39
Loss = 2.5656e-01, PNorm = 95.8748, GNorm = 4.2345, lr_0 = 9.4735e-04
Validation auc = 0.679343
Epoch 40
Validation auc = 0.680282
Epoch 41
Validation auc = 0.674648
Epoch 42
Validation auc = 0.662911
Epoch 43
Validation auc = 0.682160
Epoch 44
Loss = 2.5555e-01, PNorm = 96.0821, GNorm = 3.4399, lr_0 = 9.3925e-04
Validation auc = 0.686854
Epoch 45
Validation auc = 0.686385
Epoch 46
Validation auc = 0.665258
Epoch 47
Validation auc = 0.643662
Epoch 48
Validation auc = 0.636150
Epoch 49
Loss = 2.8385e-01, PNorm = 96.3391, GNorm = 6.2023, lr_0 = 9.3122e-04
Validation auc = 0.627700
Epoch 50
Validation auc = 0.634272
Epoch 51
Validation auc = 0.657746
Epoch 52
Validation auc = 0.696714
Epoch 53
Validation auc = 0.717371
Epoch 54
Loss = 2.9994e-01, PNorm = 96.5982, GNorm = 3.9710, lr_0 = 9.2326e-04
Validation auc = 0.719718
Epoch 55
Validation auc = 0.713146
Epoch 56
Validation auc = 0.697183
Epoch 57
Validation auc = 0.699531
Epoch 58
Validation auc = 0.698592
Epoch 59
Loss = 1.6260e-01, PNorm = 96.8528, GNorm = 3.6591, lr_0 = 9.1537e-04
Validation auc = 0.683568
Epoch 60
Validation auc = 0.692958
Epoch 61
Validation auc = 0.697183
Epoch 62
Validation auc = 0.695305
Epoch 63
Validation auc = 0.696714
Epoch 64
Loss = 2.3334e-01, PNorm = 97.0761, GNorm = 4.1925, lr_0 = 9.0754e-04
Validation auc = 0.688263
Epoch 65
Validation auc = 0.693897
Epoch 66
Validation auc = 0.693427
Epoch 67
Validation auc = 0.701408
Epoch 68
Validation auc = 0.684507
Epoch 69
Loss = 2.0532e-01, PNorm = 97.3694, GNorm = 4.0349, lr_0 = 8.9978e-04
Validation auc = 0.684038
Epoch 70
Validation auc = 0.712207
Epoch 71
Validation auc = 0.723944
Epoch 72
Validation auc = 0.711737
Epoch 73
Validation auc = 0.702347
Epoch 74
Loss = 9.2510e-02, PNorm = 97.6276, GNorm = 3.3665, lr_0 = 8.9209e-04
Validation auc = 0.701878
Epoch 75
Validation auc = 0.699531
Epoch 76
Validation auc = 0.695775
Epoch 77
Validation auc = 0.684507
Epoch 78
Validation auc = 0.694836
Epoch 79
Loss = 1.2078e-01, PNorm = 97.9000, GNorm = 3.1807, lr_0 = 8.8447e-04
Validation auc = 0.693427
Epoch 80
Validation auc = 0.704225
Epoch 81
Validation auc = 0.720188
Epoch 82
Validation auc = 0.737089
Epoch 83
Validation auc = 0.741784
Epoch 84
Loss = 2.3067e-01, PNorm = 98.1645, GNorm = 8.8752, lr_0 = 8.7691e-04
Validation auc = 0.746948
Epoch 85
Validation auc = 0.724413
Epoch 86
Validation auc = 0.698122
Epoch 87
Validation auc = 0.717840
Epoch 88
Validation auc = 0.737559
Epoch 89
Loss = 2.0599e-01, PNorm = 98.4365, GNorm = 5.0002, lr_0 = 8.6941e-04
Validation auc = 0.726291
Epoch 90
Validation auc = 0.754460
Epoch 91
Validation auc = 0.759624
Epoch 92
Validation auc = 0.741315
Epoch 93
Validation auc = 0.721127
Epoch 94
Loss = 9.9221e-02, PNorm = 98.7434, GNorm = 2.3314, lr_0 = 8.6198e-04
Validation auc = 0.694836
Epoch 95
Validation auc = 0.661502
Epoch 96
Validation auc = 0.651643
Epoch 97
Validation auc = 0.661972
Epoch 98
Validation auc = 0.689202
Epoch 99
Loss = 1.0094e-01, PNorm = 99.0789, GNorm = 2.4389, lr_0 = 8.5461e-04
Validation auc = 0.706573
Epoch 100
Validation auc = 0.709390
Epoch 101
Validation auc = 0.711737
Epoch 102
Validation auc = 0.712207
Epoch 103
Validation auc = 0.700469
Epoch 104
Loss = 1.4370e-01, PNorm = 99.3499, GNorm = 3.1217, lr_0 = 8.4730e-04
Validation auc = 0.701408
Epoch 105
Validation auc = 0.711268
Epoch 106
Validation auc = 0.719718
Epoch 107
Validation auc = 0.714085
Epoch 108
Validation auc = 0.709390
Epoch 109
Loss = 6.5713e-02, PNorm = 99.5490, GNorm = 1.8705, lr_0 = 8.4006e-04
Validation auc = 0.699531
Epoch 110
Validation auc = 0.702347
Epoch 111
Validation auc = 0.696244
Epoch 112
Validation auc = 0.694836
Epoch 113
Validation auc = 0.693427
Epoch 114
Loss = 1.2697e-01, PNorm = 99.7577, GNorm = 1.3570, lr_0 = 8.3288e-04
Validation auc = 0.688732
Epoch 115
Validation auc = 0.691549
Epoch 116
Validation auc = 0.695775
Epoch 117
Validation auc = 0.689671
Epoch 118
Validation auc = 0.695305
Epoch 119
Loss = 7.5466e-02, PNorm = 99.9648, GNorm = 1.3757, lr_0 = 8.2576e-04
Validation auc = 0.708920
Epoch 120
Validation auc = 0.722066
Epoch 121
Validation auc = 0.731925
Epoch 122
Validation auc = 0.739906
Epoch 123
Validation auc = 0.721596
Epoch 124
Loss = 1.1133e-01, PNorm = 100.1935, GNorm = 2.2740, lr_0 = 8.1870e-04
Validation auc = 0.689202
Epoch 125
Validation auc = 0.680751
Epoch 126
Validation auc = 0.700939
Epoch 127
Validation auc = 0.716432
Epoch 128
Validation auc = 0.723474
Epoch 129
Loss = 1.3335e-01, PNorm = 100.4178, GNorm = 0.8480, lr_0 = 8.1170e-04
Validation auc = 0.730516
Epoch 130
Validation auc = 0.724883
Epoch 131
Validation auc = 0.718310
Epoch 132
Validation auc = 0.718779
Epoch 133
Validation auc = 0.717371
Epoch 134
Loss = 9.0921e-02, PNorm = 100.6493, GNorm = 2.9135, lr_0 = 8.0476e-04
Validation auc = 0.711737
Epoch 135
Validation auc = 0.710798
Epoch 136
Validation auc = 0.705634
Epoch 137
Validation auc = 0.693427
Epoch 138
Validation auc = 0.694836
Epoch 139
Loss = 1.1122e-01, PNorm = 100.8203, GNorm = 0.7222, lr_0 = 7.9788e-04
Validation auc = 0.694836
Epoch 140
Validation auc = 0.706573
Epoch 141
Validation auc = 0.708920
Epoch 142
Validation auc = 0.706573
Epoch 143
Validation auc = 0.714554
Epoch 144
Loss = 1.0200e-01, PNorm = 100.9874, GNorm = 2.4409, lr_0 = 7.9106e-04
Validation auc = 0.721127
Epoch 145
Validation auc = 0.712207
Epoch 146
Validation auc = 0.699061
Epoch 147
Validation auc = 0.694836
Epoch 148
Validation auc = 0.689671
Epoch 149
Loss = 9.0754e-02, PNorm = 101.1386, GNorm = 1.9735, lr_0 = 7.8430e-04
Validation auc = 0.683568
Epoch 150
Validation auc = 0.688263
Epoch 151
Validation auc = 0.693897
Epoch 152
Validation auc = 0.707512
Epoch 153
Validation auc = 0.714085
Epoch 154
Loss = 1.9482e-01, PNorm = 101.2911, GNorm = 5.9136, lr_0 = 7.7759e-04
Validation auc = 0.721127
Epoch 155
Validation auc = 0.727230
Epoch 156
Validation auc = 0.722066
Epoch 157
Validation auc = 0.711268
Epoch 158
Validation auc = 0.706573
Epoch 159
Loss = 1.1892e-01, PNorm = 101.4867, GNorm = 1.6292, lr_0 = 7.7095e-04
Validation auc = 0.701408
Epoch 160
Validation auc = 0.696714
Epoch 161
Validation auc = 0.699061
Epoch 162
Validation auc = 0.710798
Epoch 163
Validation auc = 0.702817
Epoch 164
Loss = 9.6523e-02, PNorm = 101.6594, GNorm = 6.1460, lr_0 = 7.6436e-04
Validation auc = 0.713146
Epoch 165
Validation auc = 0.718779
Epoch 166
Validation auc = 0.714554
Epoch 167
Validation auc = 0.698122
Epoch 168
Validation auc = 0.683568
Epoch 169
Loss = 9.9505e-02, PNorm = 101.8254, GNorm = 3.4761, lr_0 = 7.5782e-04
Validation auc = 0.680751
Epoch 170
Validation auc = 0.692488
Epoch 171
Validation auc = 0.708451
Epoch 172
Validation auc = 0.719718
Epoch 173
Validation auc = 0.730986
Epoch 174
Loss = 1.6227e-01, PNorm = 102.0011, GNorm = 4.2811, lr_0 = 7.5134e-04
Validation auc = 0.729108
Epoch 175
Validation auc = 0.734742
Epoch 176
Validation auc = 0.734742
Epoch 177
Validation auc = 0.723944
Epoch 178
Validation auc = 0.719249
Epoch 179
Loss = 1.2997e-02, PNorm = 102.1768, GNorm = 0.9352, lr_0 = 7.4492e-04
Validation auc = 0.713146
Epoch 180
Validation auc = 0.705164
Epoch 181
Validation auc = 0.713615
Epoch 182
Validation auc = 0.713615
Epoch 183
Validation auc = 0.710798
Epoch 184
Loss = 7.7110e-03, PNorm = 102.3469, GNorm = 0.3289, lr_0 = 7.3855e-04
Validation auc = 0.699061
Epoch 185
Validation auc = 0.691549
Epoch 186
Validation auc = 0.683568
Epoch 187
Validation auc = 0.683568
Epoch 188
Validation auc = 0.680282
Epoch 189
Loss = 1.7693e-02, PNorm = 102.5057, GNorm = 0.7323, lr_0 = 7.3224e-04
Validation auc = 0.676995
Epoch 190
Validation auc = 0.677465
Epoch 191
Validation auc = 0.682160
Epoch 192
Validation auc = 0.684038
Epoch 193
Validation auc = 0.691080
Epoch 194
Loss = 4.1369e-02, PNorm = 102.6409, GNorm = 2.4643, lr_0 = 7.2598e-04
Validation auc = 0.700939
Epoch 195
Validation auc = 0.699061
Epoch 196
Validation auc = 0.694836
Epoch 197
Validation auc = 0.696244
Epoch 198
Validation auc = 0.698592
Epoch 199
Loss = 5.1055e-02, PNorm = 102.7596, GNorm = 0.5088, lr_0 = 7.1977e-04
Validation auc = 0.698122
Epoch 200
Validation auc = 0.699531
Epoch 201
Validation auc = 0.690610
Epoch 202
Validation auc = 0.689671
Epoch 203
Validation auc = 0.685915
Epoch 204
Loss = 1.4891e-02, PNorm = 102.9082, GNorm = 1.2913, lr_0 = 7.1362e-04
Validation auc = 0.692488
Epoch 205
Validation auc = 0.696244
Epoch 206
Validation auc = 0.699061
Epoch 207
Validation auc = 0.702347
Epoch 208
Validation auc = 0.701408
Epoch 209
Loss = 4.3780e-02, PNorm = 103.0547, GNorm = 0.7426, lr_0 = 7.0752e-04
Validation auc = 0.701878
Epoch 210
Validation auc = 0.698122
Epoch 211
Validation auc = 0.690141
Epoch 212
Validation auc = 0.677465
Epoch 213
Validation auc = 0.666667
Epoch 214
Loss = 4.8522e-02, PNorm = 103.2133, GNorm = 1.2436, lr_0 = 7.0147e-04
Validation auc = 0.657746
Epoch 215
Validation auc = 0.658216
Epoch 216
Validation auc = 0.654460
Epoch 217
Validation auc = 0.662911
Epoch 218
Validation auc = 0.671831
Epoch 219
Loss = 1.6451e-02, PNorm = 103.3565, GNorm = 0.2318, lr_0 = 6.9548e-04
Validation auc = 0.676526
Epoch 220
Validation auc = 0.684977
Epoch 221
Validation auc = 0.697183
Epoch 222
Validation auc = 0.700000
Epoch 223
Validation auc = 0.703756
Epoch 224
Loss = 1.6279e-02, PNorm = 103.5193, GNorm = 1.3090, lr_0 = 6.8953e-04
Validation auc = 0.700469
Epoch 225
Validation auc = 0.691549
Epoch 226
Validation auc = 0.680751
Epoch 227
Validation auc = 0.683568
Epoch 228
Validation auc = 0.676056
Epoch 229
Loss = 4.5263e-02, PNorm = 103.6796, GNorm = 1.9686, lr_0 = 6.8364e-04
Validation auc = 0.666667
Epoch 230
Validation auc = 0.670892
Epoch 231
Validation auc = 0.675117
Epoch 232
Validation auc = 0.675117
Epoch 233
Validation auc = 0.686385
Epoch 234
Loss = 2.8190e-02, PNorm = 103.8304, GNorm = 0.2710, lr_0 = 6.7779e-04
Validation auc = 0.690141
Epoch 235
Validation auc = 0.692958
Epoch 236
Validation auc = 0.694366
Epoch 237
Validation auc = 0.698122
Epoch 238
Validation auc = 0.697653
Epoch 239
Loss = 3.4943e-02, PNorm = 103.9694, GNorm = 0.5453, lr_0 = 6.7200e-04
Validation auc = 0.695775
Epoch 240
Validation auc = 0.684977
Epoch 241
Validation auc = 0.685446
Epoch 242
Validation auc = 0.690610
Epoch 243
Validation auc = 0.698122
Epoch 244
Loss = 1.7916e-01, PNorm = 104.0780, GNorm = 2.6466, lr_0 = 6.6625e-04
Validation auc = 0.693897
Epoch 245
Validation auc = 0.689671
Epoch 246
Validation auc = 0.693427
Epoch 247
Validation auc = 0.695305
Epoch 248
Validation auc = 0.696714
Epoch 249
Loss = 4.2071e-02, PNorm = 104.2079, GNorm = 1.0964, lr_0 = 6.6056e-04
Validation auc = 0.698122
Epoch 250
Validation auc = 0.706573
Epoch 251
Validation auc = 0.698592
Epoch 252
Validation auc = 0.695775
Epoch 253
Validation auc = 0.696714
Epoch 254
Loss = 2.2182e-02, PNorm = 104.3225, GNorm = 0.8069, lr_0 = 6.5491e-04
Validation auc = 0.691080
Epoch 255
Validation auc = 0.686385
Epoch 256
Validation auc = 0.680751
Epoch 257
Validation auc = 0.673239
Epoch 258
Validation auc = 0.679343
Epoch 259
Loss = 1.5352e-01, PNorm = 104.4347, GNorm = 0.8061, lr_0 = 6.4931e-04
Validation auc = 0.685446
Epoch 260
Validation auc = 0.691549
Epoch 261
Validation auc = 0.690141
Epoch 262
Validation auc = 0.691080
Epoch 263
Validation auc = 0.691549
Epoch 264
Loss = 9.4490e-02, PNorm = 104.5472, GNorm = 0.4855, lr_0 = 6.4376e-04
Validation auc = 0.689202
Epoch 265
Validation auc = 0.685446
Epoch 266
Validation auc = 0.679343
Epoch 267
Validation auc = 0.680751
Epoch 268
Validation auc = 0.679343
Epoch 269
Loss = 1.6243e-02, PNorm = 104.6614, GNorm = 0.3626, lr_0 = 6.3826e-04
Validation auc = 0.678404
Epoch 270
Validation auc = 0.675117
Epoch 271
Validation auc = 0.674178
Epoch 272
Validation auc = 0.675587
Epoch 273
Validation auc = 0.682160
Epoch 274
Loss = 7.2187e-03, PNorm = 104.7612, GNorm = 0.3108, lr_0 = 6.3280e-04
Validation auc = 0.680751
Epoch 275
Validation auc = 0.678404
Epoch 276
Validation auc = 0.677465
Epoch 277
Validation auc = 0.676056
Epoch 278
Validation auc = 0.671831
Epoch 279
Loss = 1.9030e-02, PNorm = 104.8261, GNorm = 0.0795, lr_0 = 6.2739e-04
Validation auc = 0.664789
Epoch 280
Validation auc = 0.662441
Epoch 281
Validation auc = 0.653991
Epoch 282
Validation auc = 0.651174
Epoch 283
Validation auc = 0.648826
Epoch 284
Loss = 3.8666e-02, PNorm = 104.8968, GNorm = 1.9260, lr_0 = 6.2203e-04
Validation auc = 0.650235
Epoch 285
Validation auc = 0.662911
Epoch 286
Validation auc = 0.667606
Epoch 287
Validation auc = 0.670423
Epoch 288
Validation auc = 0.673239
Epoch 289
Loss = 1.9181e-02, PNorm = 104.9768, GNorm = 1.8129, lr_0 = 6.1671e-04
Validation auc = 0.676995
Epoch 290
Validation auc = 0.676995
Epoch 291
Validation auc = 0.680751
Epoch 292
Validation auc = 0.682629
Epoch 293
Validation auc = 0.678404
Epoch 294
Loss = 1.6341e-02, PNorm = 105.0582, GNorm = 0.1456, lr_0 = 6.1144e-04
Validation auc = 0.679812
Epoch 295
Validation auc = 0.675117
Epoch 296
Validation auc = 0.670892
Epoch 297
Validation auc = 0.671362
Epoch 298
Validation auc = 0.671831
Epoch 299
Loss = 6.3049e-03, PNorm = 105.1309, GNorm = 0.3251, lr_0 = 6.0621e-04
Validation auc = 0.671362
Model 0 best validation auc = 0.759624 on epoch 91
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.779952
Ensemble test auc = 0.779952
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=2100, out_features=2100, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.2, inplace=False)
    (7): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 18,866,401
Moving model to cuda
Epoch 0
Validation auc = 0.695305
Epoch 1
Validation auc = 0.703756
Epoch 2
Validation auc = 0.753052
Epoch 3
Validation auc = 0.744131
Epoch 4
Loss = 1.3579e+00, PNorm = 94.3387, GNorm = 15.4384, lr_0 = 6.5000e-04
Validation auc = 0.683568
Epoch 5
Validation auc = 0.619718
Epoch 6
Validation auc = 0.671831
Epoch 7
Validation auc = 0.696244
Epoch 8
Validation auc = 0.725352
Epoch 9
Loss = 9.8084e-01, PNorm = 94.6575, GNorm = 12.4854, lr_0 = 9.9743e-04
Validation auc = 0.689202
Epoch 10
Validation auc = 0.654930
Epoch 11
Validation auc = 0.613615
Epoch 12
Validation auc = 0.628638
Epoch 13
Validation auc = 0.649765
Epoch 14
Loss = 5.4098e-01, PNorm = 94.9844, GNorm = 5.7723, lr_0 = 9.8890e-04
Validation auc = 0.678873
Epoch 15
Validation auc = 0.707981
Epoch 16
Validation auc = 0.701408
Epoch 17
Validation auc = 0.702817
Epoch 18
Validation auc = 0.705634
Epoch 19
Loss = 3.8831e-01, PNorm = 95.1897, GNorm = 4.5778, lr_0 = 9.8045e-04
Validation auc = 0.686385
Epoch 20
Validation auc = 0.663380
Epoch 21
Validation auc = 0.669014
Epoch 22
Validation auc = 0.694836
Epoch 23
Validation auc = 0.682629
Epoch 24
Loss = 3.3844e-01, PNorm = 95.3205, GNorm = 5.4408, lr_0 = 9.7207e-04
Validation auc = 0.660563
Epoch 25
Validation auc = 0.652582
Epoch 26
Validation auc = 0.668545
Epoch 27
Validation auc = 0.687324
Epoch 28
Validation auc = 0.659155
Epoch 29
Loss = 2.8479e-01, PNorm = 95.4249, GNorm = 3.5600, lr_0 = 9.6376e-04
Validation auc = 0.647418
Epoch 30
Validation auc = 0.623944
Epoch 31
Validation auc = 0.641315
Epoch 32
Validation auc = 0.676056
Epoch 33
Validation auc = 0.700939
Epoch 34
Loss = 2.4815e-01, PNorm = 95.5468, GNorm = 2.7125, lr_0 = 9.5552e-04
Validation auc = 0.710798
Epoch 35
Validation auc = 0.706573
Epoch 36
Validation auc = 0.690141
Epoch 37
Validation auc = 0.672300
Epoch 38
Validation auc = 0.640845
Epoch 39
Loss = 3.6607e-01, PNorm = 95.7058, GNorm = 4.5623, lr_0 = 9.4735e-04
Validation auc = 0.688732
Epoch 40
Validation auc = 0.721127
Epoch 41
Validation auc = 0.724883
Epoch 42
Validation auc = 0.696714
Epoch 43
Validation auc = 0.663380
Epoch 44
Loss = 3.2306e-01, PNorm = 95.9319, GNorm = 3.3391, lr_0 = 9.3925e-04
Validation auc = 0.636150
Epoch 45
Validation auc = 0.669014
Epoch 46
Validation auc = 0.670892
Epoch 47
Validation auc = 0.667136
Epoch 48
Validation auc = 0.643192
Epoch 49
Loss = 1.5793e-01, PNorm = 96.1620, GNorm = 1.3963, lr_0 = 9.3122e-04
Validation auc = 0.627230
Epoch 50
Validation auc = 0.640845
Epoch 51
Validation auc = 0.677934
Epoch 52
Validation auc = 0.691080
Epoch 53
Validation auc = 0.698122
Epoch 54
Loss = 1.2967e-01, PNorm = 96.3945, GNorm = 2.0806, lr_0 = 9.2326e-04
Validation auc = 0.699531
Epoch 55
Validation auc = 0.690141
Epoch 56
Validation auc = 0.680282
Epoch 57
Validation auc = 0.677934
Epoch 58
Validation auc = 0.720657
Epoch 59
Loss = 4.2650e-01, PNorm = 96.6072, GNorm = 8.0920, lr_0 = 9.1537e-04
Validation auc = 0.715023
Epoch 60
Validation auc = 0.686385
Epoch 61
Validation auc = 0.673709
Epoch 62
Validation auc = 0.668545
Epoch 63
Validation auc = 0.670423
Epoch 64
Loss = 1.7764e-01, PNorm = 96.8524, GNorm = 2.9629, lr_0 = 9.0754e-04
Validation auc = 0.685446
Epoch 65
Validation auc = 0.698122
Epoch 66
Validation auc = 0.710798
Epoch 67
Validation auc = 0.706573
Epoch 68
Validation auc = 0.716432
Epoch 69
Loss = 9.6873e-02, PNorm = 97.0947, GNorm = 1.5101, lr_0 = 8.9978e-04
Validation auc = 0.692958
Epoch 70
Validation auc = 0.693897
Epoch 71
Validation auc = 0.683568
Epoch 72
Validation auc = 0.688732
Epoch 73
Validation auc = 0.699531
Epoch 74
Loss = 1.3635e-01, PNorm = 97.3544, GNorm = 1.6820, lr_0 = 8.9209e-04
Validation auc = 0.695775
Epoch 75
Validation auc = 0.705164
Epoch 76
Validation auc = 0.713615
Epoch 77
Validation auc = 0.726291
Epoch 78
Validation auc = 0.720657
Epoch 79
Loss = 1.4834e-01, PNorm = 97.6373, GNorm = 3.3108, lr_0 = 8.8447e-04
Validation auc = 0.709859
Epoch 80
Validation auc = 0.700939
Epoch 81
Validation auc = 0.701408
Epoch 82
Validation auc = 0.696244
Epoch 83
Validation auc = 0.692958
Epoch 84
Loss = 1.4479e-01, PNorm = 97.8561, GNorm = 1.8110, lr_0 = 8.7691e-04
Validation auc = 0.694366
Epoch 85
Validation auc = 0.697183
Epoch 86
Validation auc = 0.715023
Epoch 87
Validation auc = 0.736150
Epoch 88
Validation auc = 0.723944
Epoch 89
Loss = 1.6598e-01, PNorm = 98.0961, GNorm = 4.2860, lr_0 = 8.6941e-04
Validation auc = 0.720657
Epoch 90
Validation auc = 0.721596
Epoch 91
Validation auc = 0.719718
Epoch 92
Validation auc = 0.718779
Epoch 93
Validation auc = 0.726291
Epoch 94
Loss = 3.0938e-01, PNorm = 98.3436, GNorm = 4.7218, lr_0 = 8.6198e-04
Validation auc = 0.726291
Epoch 95
Validation auc = 0.731925
Epoch 96
Validation auc = 0.724883
Epoch 97
Validation auc = 0.721596
Epoch 98
Validation auc = 0.725822
Epoch 99
Loss = 9.2255e-02, PNorm = 98.6000, GNorm = 1.8991, lr_0 = 8.5461e-04
Validation auc = 0.725822
Epoch 100
Validation auc = 0.719718
Epoch 101
Validation auc = 0.713615
Epoch 102
Validation auc = 0.708920
Epoch 103
Validation auc = 0.699531
Epoch 104
Loss = 5.7643e-02, PNorm = 98.8173, GNorm = 0.6155, lr_0 = 8.4730e-04
Validation auc = 0.694836
Epoch 105
Validation auc = 0.705634
Epoch 106
Validation auc = 0.707042
Epoch 107
Validation auc = 0.716432
Epoch 108
Validation auc = 0.724413
Epoch 109
Loss = 7.4458e-02, PNorm = 99.0347, GNorm = 1.6092, lr_0 = 8.4006e-04
Validation auc = 0.727230
Epoch 110
Validation auc = 0.723944
Epoch 111
Validation auc = 0.719718
Epoch 112
Validation auc = 0.719718
Epoch 113
Validation auc = 0.722535
Epoch 114
Loss = 6.8247e-02, PNorm = 99.2503, GNorm = 1.3400, lr_0 = 8.3288e-04
Validation auc = 0.716901
Epoch 115
Validation auc = 0.714554
Epoch 116
Validation auc = 0.712207
Epoch 117
Validation auc = 0.707512
Epoch 118
Validation auc = 0.709390
Epoch 119
Loss = 2.2796e-01, PNorm = 99.4671, GNorm = 2.4580, lr_0 = 8.2576e-04
Validation auc = 0.703756
Epoch 120
Validation auc = 0.707512
Epoch 121
Validation auc = 0.717371
Epoch 122
Validation auc = 0.721596
Epoch 123
Validation auc = 0.724413
Epoch 124
Loss = 1.8846e-01, PNorm = 99.6968, GNorm = 0.9152, lr_0 = 8.1870e-04
Validation auc = 0.711268
Epoch 125
Validation auc = 0.696714
Epoch 126
Validation auc = 0.677465
Epoch 127
Validation auc = 0.677934
Epoch 128
Validation auc = 0.693427
Epoch 129
Loss = 8.8889e-02, PNorm = 99.9247, GNorm = 1.4487, lr_0 = 8.1170e-04
Validation auc = 0.710329
Epoch 130
Validation auc = 0.716901
Epoch 131
Validation auc = 0.715962
Epoch 132
Validation auc = 0.717840
Epoch 133
Validation auc = 0.718310
Epoch 134
Loss = 5.2109e-02, PNorm = 100.1644, GNorm = 1.0912, lr_0 = 8.0476e-04
Validation auc = 0.722535
Epoch 135
Validation auc = 0.718779
Epoch 136
Validation auc = 0.718779
Epoch 137
Validation auc = 0.714554
Epoch 138
Validation auc = 0.713146
Epoch 139
Loss = 4.8184e-02, PNorm = 100.3956, GNorm = 1.0984, lr_0 = 7.9788e-04
Validation auc = 0.714554
Epoch 140
Validation auc = 0.713615
Epoch 141
Validation auc = 0.715962
Epoch 142
Validation auc = 0.719718
Epoch 143
Validation auc = 0.719249
Epoch 144
Loss = 2.3973e-02, PNorm = 100.6253, GNorm = 0.8975, lr_0 = 7.9106e-04
Validation auc = 0.717371
Epoch 145
Validation auc = 0.715962
Epoch 146
Validation auc = 0.717840
Epoch 147
Validation auc = 0.715023
Epoch 148
Validation auc = 0.716432
Epoch 149
Loss = 9.5176e-02, PNorm = 100.7969, GNorm = 3.7962, lr_0 = 7.8430e-04
Validation auc = 0.718310
Epoch 150
Validation auc = 0.716901
Epoch 151
Validation auc = 0.721596
Epoch 152
Validation auc = 0.724413
Epoch 153
Validation auc = 0.723944
Epoch 154
Loss = 5.2938e-02, PNorm = 100.9350, GNorm = 2.6365, lr_0 = 7.7759e-04
Validation auc = 0.715493
Epoch 155
Validation auc = 0.716432
Epoch 156
Validation auc = 0.720188
Epoch 157
Validation auc = 0.729108
Epoch 158
Validation auc = 0.728169
Epoch 159
Loss = 5.2072e-02, PNorm = 101.1247, GNorm = 1.8325, lr_0 = 7.7095e-04
Validation auc = 0.723474
Epoch 160
Validation auc = 0.715962
Epoch 161
Validation auc = 0.698592
Epoch 162
Validation auc = 0.695775
Epoch 163
Validation auc = 0.692958
Epoch 164
Loss = 6.7002e-02, PNorm = 101.4085, GNorm = 1.1945, lr_0 = 7.6436e-04
Validation auc = 0.696244
Epoch 165
Validation auc = 0.703286
Epoch 166
Validation auc = 0.710798
Epoch 167
Validation auc = 0.715493
Epoch 168
Validation auc = 0.722066
Epoch 169
Loss = 2.7933e-02, PNorm = 101.7126, GNorm = 0.9550, lr_0 = 7.5782e-04
Validation auc = 0.718779
Epoch 170
Validation auc = 0.726761
Epoch 171
Validation auc = 0.722535
Epoch 172
Validation auc = 0.722535
Epoch 173
Validation auc = 0.717371
Epoch 174
Loss = 6.2424e-02, PNorm = 101.9196, GNorm = 0.3248, lr_0 = 7.5134e-04
Validation auc = 0.717840
Epoch 175
Validation auc = 0.718310
Epoch 176
Validation auc = 0.723005
Epoch 177
Validation auc = 0.719718
Epoch 178
Validation auc = 0.721596
Epoch 179
Loss = 6.5429e-02, PNorm = 102.1230, GNorm = 2.2234, lr_0 = 7.4492e-04
Validation auc = 0.721127
Epoch 180
Validation auc = 0.725822
Epoch 181
Validation auc = 0.721127
Epoch 182
Validation auc = 0.719718
Epoch 183
Validation auc = 0.722535
Epoch 184
Loss = 4.2566e-02, PNorm = 102.3084, GNorm = 1.8152, lr_0 = 7.3855e-04
Validation auc = 0.727230
Epoch 185
Validation auc = 0.726761
Epoch 186
Validation auc = 0.723944
Epoch 187
Validation auc = 0.729108
Epoch 188
Validation auc = 0.734272
Epoch 189
Loss = 5.7169e-02, PNorm = 102.4541, GNorm = 2.5268, lr_0 = 7.3224e-04
Validation auc = 0.735681
Epoch 190
Validation auc = 0.730986
Epoch 191
Validation auc = 0.729108
Epoch 192
Validation auc = 0.731925
Epoch 193
Validation auc = 0.736150
Epoch 194
Loss = 2.8689e-02, PNorm = 102.6168, GNorm = 1.6156, lr_0 = 7.2598e-04
Validation auc = 0.736150
Epoch 195
Validation auc = 0.725822
Epoch 196
Validation auc = 0.721596
Epoch 197
Validation auc = 0.715493
Epoch 198
Validation auc = 0.697653
Epoch 199
Loss = 5.7032e-02, PNorm = 102.8284, GNorm = 1.0891, lr_0 = 7.1977e-04
Validation auc = 0.689202
Epoch 200
Validation auc = 0.678404
Epoch 201
Validation auc = 0.679343
Epoch 202
Validation auc = 0.707042
Epoch 203
Validation auc = 0.722066
Epoch 204
Loss = 2.3012e-02, PNorm = 103.0694, GNorm = 1.1223, lr_0 = 7.1362e-04
Validation auc = 0.723944
Epoch 205
Validation auc = 0.723005
Epoch 206
Validation auc = 0.725352
Epoch 207
Validation auc = 0.726291
Epoch 208
Validation auc = 0.725822
Epoch 209
Loss = 1.0481e-01, PNorm = 103.2527, GNorm = 2.1973, lr_0 = 7.0752e-04
Validation auc = 0.725352
Epoch 210
Validation auc = 0.727230
Epoch 211
Validation auc = 0.727230
Epoch 212
Validation auc = 0.727700
Epoch 213
Validation auc = 0.728638
Epoch 214
Loss = 4.8135e-02, PNorm = 103.4134, GNorm = 1.0588, lr_0 = 7.0147e-04
Validation auc = 0.730047
Epoch 215
Validation auc = 0.737089
Epoch 216
Validation auc = 0.743192
Epoch 217
Validation auc = 0.747418
Epoch 218
Validation auc = 0.748826
Epoch 219
Loss = 1.9965e-02, PNorm = 103.5653, GNorm = 0.5068, lr_0 = 6.9548e-04
Validation auc = 0.750704
Epoch 220
Validation auc = 0.749765
Epoch 221
Validation auc = 0.743662
Epoch 222
Validation auc = 0.738028
Epoch 223
Validation auc = 0.729577
Epoch 224
Loss = 9.4432e-02, PNorm = 103.6800, GNorm = 2.4564, lr_0 = 6.8953e-04
Validation auc = 0.724413
Epoch 225
Validation auc = 0.719249
Epoch 226
Validation auc = 0.710798
Epoch 227
Validation auc = 0.709390
Epoch 228
Validation auc = 0.712207
Epoch 229
Loss = 7.8517e-03, PNorm = 103.8078, GNorm = 0.1381, lr_0 = 6.8364e-04
Validation auc = 0.711268
Epoch 230
Validation auc = 0.710329
Epoch 231
Validation auc = 0.707512
Epoch 232
Validation auc = 0.707512
Epoch 233
Validation auc = 0.707042
Epoch 234
Loss = 6.1968e-02, PNorm = 103.9227, GNorm = 1.6560, lr_0 = 6.7779e-04
Validation auc = 0.693427
Epoch 235
Validation auc = 0.685446
Epoch 236
Validation auc = 0.680751
Epoch 237
Validation auc = 0.690141
Epoch 238
Validation auc = 0.693897
Epoch 239
Loss = 1.5919e-02, PNorm = 104.0340, GNorm = 1.8227, lr_0 = 6.7200e-04
Validation auc = 0.703286
Epoch 240
Validation auc = 0.705634
Epoch 241
Validation auc = 0.701878
Epoch 242
Validation auc = 0.700939
Epoch 243
Validation auc = 0.705164
Epoch 244
Loss = 7.4277e-02, PNorm = 104.1866, GNorm = 1.4590, lr_0 = 6.6625e-04
Validation auc = 0.709390
Epoch 245
Validation auc = 0.707512
Epoch 246
Validation auc = 0.704225
Epoch 247
Validation auc = 0.702817
Epoch 248
Validation auc = 0.701878
Epoch 249
Loss = 2.3254e-02, PNorm = 104.3225, GNorm = 0.1693, lr_0 = 6.6056e-04
Validation auc = 0.699061
Epoch 250
Validation auc = 0.697653
Epoch 251
Validation auc = 0.699061
Epoch 252
Validation auc = 0.699061
Epoch 253
Validation auc = 0.703756
Epoch 254
Loss = 2.5232e-02, PNorm = 104.4662, GNorm = 1.2372, lr_0 = 6.5491e-04
Validation auc = 0.708451
Epoch 255
Validation auc = 0.708451
Epoch 256
Validation auc = 0.717840
Epoch 257
Validation auc = 0.724413
Epoch 258
Validation auc = 0.718779
Epoch 259
Loss = 7.9265e-03, PNorm = 104.6326, GNorm = 0.7006, lr_0 = 6.4931e-04
Validation auc = 0.710798
Epoch 260
Validation auc = 0.708920
Epoch 261
Validation auc = 0.713615
Epoch 262
Validation auc = 0.714554
Epoch 263
Validation auc = 0.717371
Epoch 264
Loss = 1.5677e-02, PNorm = 104.7947, GNorm = 0.4635, lr_0 = 6.4376e-04
Validation auc = 0.720188
Epoch 265
Validation auc = 0.720657
Epoch 266
Validation auc = 0.718779
Epoch 267
Validation auc = 0.727700
Epoch 268
Validation auc = 0.719249
Epoch 269
Loss = 4.4122e-02, PNorm = 104.9437, GNorm = 3.9705, lr_0 = 6.3826e-04
Validation auc = 0.693897
Epoch 270
Validation auc = 0.685446
Epoch 271
Validation auc = 0.674178
Epoch 272
Validation auc = 0.668075
Epoch 273
Validation auc = 0.668545
Epoch 274
Loss = 1.5353e-02, PNorm = 105.0618, GNorm = 1.0473, lr_0 = 6.3280e-04
Validation auc = 0.678873
Epoch 275
Validation auc = 0.690610
Epoch 276
Validation auc = 0.690141
Epoch 277
Validation auc = 0.689202
Epoch 278
Validation auc = 0.689202
Epoch 279
Loss = 8.4122e-03, PNorm = 105.1748, GNorm = 0.5760, lr_0 = 6.2739e-04
Validation auc = 0.691549
Epoch 280
Validation auc = 0.686854
Epoch 281
Validation auc = 0.689202
Epoch 282
Validation auc = 0.691549
Epoch 283
Validation auc = 0.688263
Epoch 284
Loss = 6.7165e-02, PNorm = 105.2865, GNorm = 0.9166, lr_0 = 6.2203e-04
Validation auc = 0.682629
Epoch 285
Validation auc = 0.678873
Epoch 286
Validation auc = 0.679343
Epoch 287
Validation auc = 0.678404
Epoch 288
Validation auc = 0.679812
Epoch 289
Loss = 1.1260e-02, PNorm = 105.4546, GNorm = 1.1745, lr_0 = 6.1671e-04
Validation auc = 0.679343
Epoch 290
Validation auc = 0.674648
Epoch 291
Validation auc = 0.677934
Epoch 292
Validation auc = 0.686385
Epoch 293
Validation auc = 0.698122
Epoch 294
Loss = 9.6872e-03, PNorm = 105.5788, GNorm = 0.8392, lr_0 = 6.1144e-04
Validation auc = 0.707042
Epoch 295
Validation auc = 0.708920
Epoch 296
Validation auc = 0.713146
Epoch 297
Validation auc = 0.712207
Epoch 298
Validation auc = 0.711268
Epoch 299
Loss = 7.0627e-03, PNorm = 105.6589, GNorm = 0.2164, lr_0 = 6.0621e-04
Validation auc = 0.712676
Model 0 best validation auc = 0.753052 on epoch 2
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.860325
Ensemble test auc = 0.860325
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=2100, out_features=2100, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.2, inplace=False)
    (7): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 18,866,401
Moving model to cuda
Epoch 0
Validation auc = 0.652113
Epoch 1
Validation auc = 0.662441
Epoch 2
Validation auc = 0.706573
Epoch 3
Validation auc = 0.701408
Epoch 4
Loss = 1.2137e+00, PNorm = 94.3457, GNorm = 18.8413, lr_0 = 6.5000e-04
Validation auc = 0.626291
Epoch 5
Validation auc = 0.574883
Epoch 6
Validation auc = 0.661502
Epoch 7
Validation auc = 0.712676
Epoch 8
Validation auc = 0.693427
Epoch 9
Loss = 1.2969e+00, PNorm = 94.6777, GNorm = 8.7624, lr_0 = 9.9743e-04
Validation auc = 0.686385
Epoch 10
Validation auc = 0.676056
Epoch 11
Validation auc = 0.624883
Epoch 12
Validation auc = 0.564319
Epoch 13
Validation auc = 0.723005
Epoch 14
Loss = 6.7806e-01, PNorm = 95.0299, GNorm = 8.0500, lr_0 = 9.8890e-04
Validation auc = 0.757746
Epoch 15
Validation auc = 0.745540
Epoch 16
Validation auc = 0.709859
Epoch 17
Validation auc = 0.673709
Epoch 18
Validation auc = 0.647887
Epoch 19
Loss = 5.8554e-01, PNorm = 95.2632, GNorm = 9.3978, lr_0 = 9.8045e-04
Validation auc = 0.680751
Epoch 20
Validation auc = 0.716432
Epoch 21
Validation auc = 0.747887
Epoch 22
Validation auc = 0.728638
Epoch 23
Validation auc = 0.706103
Epoch 24
Loss = 4.2461e-01, PNorm = 95.4141, GNorm = 7.3692, lr_0 = 9.7207e-04
Validation auc = 0.652113
Epoch 25
Validation auc = 0.662911
Epoch 26
Validation auc = 0.709390
Epoch 27
Validation auc = 0.728638
Epoch 28
Validation auc = 0.732394
Epoch 29
Loss = 3.5844e-01, PNorm = 95.5430, GNorm = 4.5772, lr_0 = 9.6376e-04
Validation auc = 0.717371
Epoch 30
Validation auc = 0.700939
Epoch 31
Validation auc = 0.699061
Epoch 32
Validation auc = 0.659624
Epoch 33
Validation auc = 0.651643
Epoch 34
Loss = 2.6702e-01, PNorm = 95.7034, GNorm = 3.0087, lr_0 = 9.5552e-04
Validation auc = 0.689202
Epoch 35
Validation auc = 0.709390
Epoch 36
Validation auc = 0.708451
Epoch 37
Validation auc = 0.672300
Epoch 38
Validation auc = 0.675117
Epoch 39
Loss = 4.8001e-01, PNorm = 95.9175, GNorm = 4.3182, lr_0 = 9.4735e-04
Validation auc = 0.683099
Epoch 40
Validation auc = 0.678873
Epoch 41
Validation auc = 0.687793
Epoch 42
Validation auc = 0.700000
Epoch 43
Validation auc = 0.713146
Epoch 44
Loss = 2.3569e-01, PNorm = 96.1533, GNorm = 3.8338, lr_0 = 9.3925e-04
Validation auc = 0.696244
Epoch 45
Validation auc = 0.680751
Epoch 46
Validation auc = 0.688263
Epoch 47
Validation auc = 0.695775
Epoch 48
Validation auc = 0.700939
Epoch 49
Loss = 1.8474e-01, PNorm = 96.3844, GNorm = 5.0067, lr_0 = 9.3122e-04
Validation auc = 0.700000
Epoch 50
Validation auc = 0.703286
Epoch 51
Validation auc = 0.709859
Epoch 52
Validation auc = 0.735211
Epoch 53
Validation auc = 0.733333
Epoch 54
Loss = 3.0973e-01, PNorm = 96.6315, GNorm = 4.4663, lr_0 = 9.2326e-04
Validation auc = 0.717371
Epoch 55
Validation auc = 0.707981
Epoch 56
Validation auc = 0.714554
Epoch 57
Validation auc = 0.724413
Epoch 58
Validation auc = 0.731925
Epoch 59
Loss = 1.2753e-01, PNorm = 96.8580, GNorm = 3.3366, lr_0 = 9.1537e-04
Validation auc = 0.731455
Epoch 60
Validation auc = 0.724883
Epoch 61
Validation auc = 0.720657
Epoch 62
Validation auc = 0.707042
Epoch 63
Validation auc = 0.725822
Epoch 64
Loss = 8.9094e-02, PNorm = 97.0971, GNorm = 1.0754, lr_0 = 9.0754e-04
Validation auc = 0.739906
Epoch 65
Validation auc = 0.737559
Epoch 66
Validation auc = 0.743662
Epoch 67
Validation auc = 0.742723
Epoch 68
Validation auc = 0.731455
Epoch 69
Loss = 2.0909e-01, PNorm = 97.3576, GNorm = 4.8270, lr_0 = 8.9978e-04
Validation auc = 0.713146
Epoch 70
Validation auc = 0.720188
Epoch 71
Validation auc = 0.703756
Epoch 72
Validation auc = 0.676995
Epoch 73
Validation auc = 0.670892
Epoch 74
Loss = 1.9621e-01, PNorm = 97.6385, GNorm = 3.4840, lr_0 = 8.9209e-04
Validation auc = 0.682629
Epoch 75
Validation auc = 0.697183
Epoch 76
Validation auc = 0.700939
Epoch 77
Validation auc = 0.698592
Epoch 78
Validation auc = 0.704225
Epoch 79
Loss = 6.3245e-02, PNorm = 97.9791, GNorm = 1.8770, lr_0 = 8.8447e-04
Validation auc = 0.721127
Epoch 80
Validation auc = 0.706103
Epoch 81
Validation auc = 0.697653
Epoch 82
Validation auc = 0.699531
Epoch 83
Validation auc = 0.683099
Epoch 84
Loss = 2.1227e-01, PNorm = 98.2751, GNorm = 4.2627, lr_0 = 8.7691e-04
Validation auc = 0.712676
Epoch 85
Validation auc = 0.742254
Epoch 86
Validation auc = 0.740376
Epoch 87
Validation auc = 0.738498
Epoch 88
Validation auc = 0.745070
Epoch 89
Loss = 7.3543e-02, PNorm = 98.5450, GNorm = 2.1438, lr_0 = 8.6941e-04
Validation auc = 0.748357
Epoch 90
Validation auc = 0.748826
Epoch 91
Validation auc = 0.734742
Epoch 92
Validation auc = 0.726291
Epoch 93
Validation auc = 0.724413
Epoch 94
Loss = 1.7448e-01, PNorm = 98.7912, GNorm = 3.4672, lr_0 = 8.6198e-04
Validation auc = 0.731925
Epoch 95
Validation auc = 0.735211
Epoch 96
Validation auc = 0.741315
Epoch 97
Validation auc = 0.737089
Epoch 98
Validation auc = 0.736150
Epoch 99
Loss = 9.9975e-02, PNorm = 99.0785, GNorm = 3.2828, lr_0 = 8.5461e-04
Validation auc = 0.738498
Epoch 100
Validation auc = 0.745540
Epoch 101
Validation auc = 0.740845
Epoch 102
Validation auc = 0.745070
Epoch 103
Validation auc = 0.740376
Epoch 104
Loss = 1.4000e-01, PNorm = 99.3799, GNorm = 2.2271, lr_0 = 8.4730e-04
Validation auc = 0.730047
Epoch 105
Validation auc = 0.715493
Epoch 106
Validation auc = 0.722066
Epoch 107
Validation auc = 0.723944
Epoch 108
Validation auc = 0.733803
Epoch 109
Loss = 2.5729e-02, PNorm = 99.6270, GNorm = 0.7864, lr_0 = 8.4006e-04
Validation auc = 0.740845
Epoch 110
Validation auc = 0.740845
Epoch 111
Validation auc = 0.734742
Epoch 112
Validation auc = 0.731925
Epoch 113
Validation auc = 0.731925
Epoch 114
Loss = 1.3646e-01, PNorm = 99.8691, GNorm = 3.5372, lr_0 = 8.3288e-04
Validation auc = 0.729577
Epoch 115
Validation auc = 0.727700
Epoch 116
Validation auc = 0.728638
Epoch 117
Validation auc = 0.731925
Epoch 118
Validation auc = 0.725822
Epoch 119
Loss = 1.0867e-01, PNorm = 100.1163, GNorm = 5.0336, lr_0 = 8.2576e-04
Validation auc = 0.725822
Epoch 120
Validation auc = 0.725822
Epoch 121
Validation auc = 0.719249
Epoch 122
Validation auc = 0.715962
Epoch 123
Validation auc = 0.712676
Epoch 124
Loss = 9.3434e-02, PNorm = 100.3651, GNorm = 2.2199, lr_0 = 8.1870e-04
Validation auc = 0.721596
Epoch 125
Validation auc = 0.722066
Epoch 126
Validation auc = 0.720188
Epoch 127
Validation auc = 0.717371
Epoch 128
Validation auc = 0.707042
Epoch 129
Loss = 3.6292e-02, PNorm = 100.5965, GNorm = 1.2077, lr_0 = 8.1170e-04
Validation auc = 0.704695
Epoch 130
Validation auc = 0.704225
Epoch 131
Validation auc = 0.694366
Epoch 132
Validation auc = 0.684038
Epoch 133
Validation auc = 0.695775
Epoch 134
Loss = 8.9741e-02, PNorm = 100.8387, GNorm = 3.3622, lr_0 = 8.0476e-04
Validation auc = 0.716432
Epoch 135
Validation auc = 0.735681
Epoch 136
Validation auc = 0.742723
Epoch 137
Validation auc = 0.749765
Epoch 138
Validation auc = 0.743662
Epoch 139
Loss = 7.1663e-02, PNorm = 101.0863, GNorm = 1.1507, lr_0 = 7.9788e-04
Validation auc = 0.719249
Epoch 140
Validation auc = 0.706103
Epoch 141
Validation auc = 0.695305
Epoch 142
Validation auc = 0.708920
Epoch 143
Validation auc = 0.719249
Epoch 144
Loss = 3.1139e-02, PNorm = 101.3477, GNorm = 1.2870, lr_0 = 7.9106e-04
Validation auc = 0.715493
Epoch 145
Validation auc = 0.702817
Epoch 146
Validation auc = 0.694366
Epoch 147
Validation auc = 0.688732
Epoch 148
Validation auc = 0.699061
Epoch 149
Loss = 6.2834e-02, PNorm = 101.5683, GNorm = 1.3165, lr_0 = 7.8430e-04
Validation auc = 0.707512
Epoch 150
Validation auc = 0.707512
Epoch 151
Validation auc = 0.706573
Epoch 152
Validation auc = 0.696244
Epoch 153
Validation auc = 0.682160
Epoch 154
Loss = 4.5963e-02, PNorm = 101.8179, GNorm = 1.1631, lr_0 = 7.7759e-04
Validation auc = 0.670892
Epoch 155
Validation auc = 0.662441
Epoch 156
Validation auc = 0.659624
Epoch 157
Validation auc = 0.666667
Epoch 158
Validation auc = 0.675117
Epoch 159
Loss = 1.5321e-02, PNorm = 102.0697, GNorm = 0.4196, lr_0 = 7.7095e-04
Validation auc = 0.673239
Epoch 160
Validation auc = 0.679812
Epoch 161
Validation auc = 0.683568
Epoch 162
Validation auc = 0.682160
Epoch 163
Validation auc = 0.671362
Epoch 164
Loss = 1.0724e-01, PNorm = 102.2696, GNorm = 0.5735, lr_0 = 7.6436e-04
Validation auc = 0.671831
Epoch 165
Validation auc = 0.667606
Epoch 166
Validation auc = 0.669953
Epoch 167
Validation auc = 0.668075
Epoch 168
Validation auc = 0.670892
Epoch 169
Loss = 5.9836e-02, PNorm = 102.4549, GNorm = 3.5754, lr_0 = 7.5782e-04
Validation auc = 0.679343
Epoch 170
Validation auc = 0.678404
Epoch 171
Validation auc = 0.681690
Epoch 172
Validation auc = 0.683568
Epoch 173
Validation auc = 0.691080
Epoch 174
Loss = 1.1103e-01, PNorm = 102.6112, GNorm = 3.1702, lr_0 = 7.5134e-04
Validation auc = 0.687324
Epoch 175
Validation auc = 0.679343
Epoch 176
Validation auc = 0.669484
Epoch 177
Validation auc = 0.662441
Epoch 178
Validation auc = 0.658216
Epoch 179
Loss = 4.4912e-02, PNorm = 102.7736, GNorm = 3.9610, lr_0 = 7.4492e-04
Validation auc = 0.663850
Epoch 180
Validation auc = 0.658216
Epoch 181
Validation auc = 0.657277
Epoch 182
Validation auc = 0.648826
Epoch 183
Validation auc = 0.648357
Epoch 184
Loss = 3.7844e-02, PNorm = 102.9024, GNorm = 1.1189, lr_0 = 7.3855e-04
Validation auc = 0.653521
Epoch 185
Validation auc = 0.658685
Epoch 186
Validation auc = 0.660094
Epoch 187
Validation auc = 0.658685
Epoch 188
Validation auc = 0.661033
Epoch 189
Loss = 1.4475e-02, PNorm = 103.0510, GNorm = 1.0592, lr_0 = 7.3224e-04
Validation auc = 0.658216
Epoch 190
Validation auc = 0.664789
Epoch 191
Validation auc = 0.669014
Epoch 192
Validation auc = 0.671831
Epoch 193
Validation auc = 0.667136
Epoch 194
Loss = 3.8485e-02, PNorm = 103.1669, GNorm = 1.7438, lr_0 = 7.2598e-04
Validation auc = 0.667606
Epoch 195
Validation auc = 0.672770
Epoch 196
Validation auc = 0.675587
Epoch 197
Validation auc = 0.671362
Epoch 198
Validation auc = 0.663850
Epoch 199
Loss = 2.4906e-02, PNorm = 103.2762, GNorm = 1.2669, lr_0 = 7.1977e-04
Validation auc = 0.657277
Epoch 200
Validation auc = 0.652582
Epoch 201
Validation auc = 0.641315
Epoch 202
Validation auc = 0.637559
Epoch 203
Validation auc = 0.637559
Epoch 204
Loss = 1.6070e-02, PNorm = 103.3898, GNorm = 1.3205, lr_0 = 7.1362e-04
Validation auc = 0.639906
Epoch 205
Validation auc = 0.636620
Epoch 206
Validation auc = 0.647887
Epoch 207
Validation auc = 0.652582
Epoch 208
Validation auc = 0.655869
Epoch 209
Loss = 3.8647e-02, PNorm = 103.5144, GNorm = 0.4872, lr_0 = 7.0752e-04
Validation auc = 0.671362
Epoch 210
Validation auc = 0.677934
Epoch 211
Validation auc = 0.674648
Epoch 212
Validation auc = 0.666667
Epoch 213
Validation auc = 0.664789
Epoch 214
Loss = 9.7556e-02, PNorm = 103.6476, GNorm = 2.3960, lr_0 = 7.0147e-04
Validation auc = 0.666197
Epoch 215
Validation auc = 0.665728
Epoch 216
Validation auc = 0.662441
Epoch 217
Validation auc = 0.656808
Epoch 218
Validation auc = 0.651643
Epoch 219
Loss = 1.6766e-02, PNorm = 103.8092, GNorm = 1.0113, lr_0 = 6.9548e-04
Validation auc = 0.653521
Epoch 220
Validation auc = 0.651174
Epoch 221
Validation auc = 0.662911
Epoch 222
Validation auc = 0.680751
Epoch 223
Validation auc = 0.686854
Epoch 224
Loss = 2.9317e-02, PNorm = 103.9429, GNorm = 1.2220, lr_0 = 6.8953e-04
Validation auc = 0.699061
Epoch 225
Validation auc = 0.702347
Epoch 226
Validation auc = 0.707512
Epoch 227
Validation auc = 0.711737
Epoch 228
Validation auc = 0.702817
Epoch 229
Loss = 2.3573e-02, PNorm = 104.1324, GNorm = 1.9299, lr_0 = 6.8364e-04
Validation auc = 0.688263
Epoch 230
Validation auc = 0.677934
Epoch 231
Validation auc = 0.676526
Epoch 232
Validation auc = 0.676526
Epoch 233
Validation auc = 0.678873
Epoch 234
Loss = 3.4677e-02, PNorm = 104.3524, GNorm = 1.9760, lr_0 = 6.7779e-04
Validation auc = 0.674648
Epoch 235
Validation auc = 0.678404
Epoch 236
Validation auc = 0.690141
Epoch 237
Validation auc = 0.700939
Epoch 238
Validation auc = 0.709390
Epoch 239
Loss = 4.3425e-02, PNorm = 104.5475, GNorm = 0.5665, lr_0 = 6.7200e-04
Validation auc = 0.707042
Epoch 240
Validation auc = 0.703756
Epoch 241
Validation auc = 0.699061
Epoch 242
Validation auc = 0.694366
Epoch 243
Validation auc = 0.695305
Epoch 244
Loss = 1.0138e-01, PNorm = 104.7281, GNorm = 4.0535, lr_0 = 6.6625e-04
Validation auc = 0.701408
Epoch 245
Validation auc = 0.702817
Epoch 246
Validation auc = 0.707981
Epoch 247
Validation auc = 0.712676
Epoch 248
Validation auc = 0.706103
Epoch 249
Loss = 2.6424e-02, PNorm = 104.9068, GNorm = 1.7940, lr_0 = 6.6056e-04
Validation auc = 0.691080
Epoch 250
Validation auc = 0.688732
Epoch 251
Validation auc = 0.681221
Epoch 252
Validation auc = 0.672300
Epoch 253
Validation auc = 0.670423
Epoch 254
Loss = 3.6741e-02, PNorm = 105.0572, GNorm = 2.2958, lr_0 = 6.5491e-04
Validation auc = 0.669953
Epoch 255
Validation auc = 0.673239
Epoch 256
Validation auc = 0.681690
Epoch 257
Validation auc = 0.683568
Epoch 258
Validation auc = 0.687324
Epoch 259
Loss = 1.0566e-01, PNorm = 105.2287, GNorm = 1.5498, lr_0 = 6.4931e-04
Validation auc = 0.692019
Epoch 260
Validation auc = 0.689202
Epoch 261
Validation auc = 0.687324
Epoch 262
Validation auc = 0.688263
Epoch 263
Validation auc = 0.692019
Epoch 264
Loss = 2.0091e-03, PNorm = 105.3934, GNorm = 0.0980, lr_0 = 6.4376e-04
Validation auc = 0.695775
Epoch 265
Validation auc = 0.699531
Epoch 266
Validation auc = 0.703286
Epoch 267
Validation auc = 0.701408
Epoch 268
Validation auc = 0.700939
Epoch 269
Loss = 5.6415e-03, PNorm = 105.4910, GNorm = 0.3905, lr_0 = 6.3826e-04
Validation auc = 0.696714
Epoch 270
Validation auc = 0.689671
Epoch 271
Validation auc = 0.685915
Epoch 272
Validation auc = 0.681690
Epoch 273
Validation auc = 0.678873
Epoch 274
Loss = 1.0478e-01, PNorm = 105.5917, GNorm = 3.0030, lr_0 = 6.3280e-04
Validation auc = 0.676995
Epoch 275
Validation auc = 0.686854
Epoch 276
Validation auc = 0.699061
Epoch 277
Validation auc = 0.707981
Epoch 278
Validation auc = 0.714085
Epoch 279
Loss = 8.1638e-02, PNorm = 105.6737, GNorm = 4.3263, lr_0 = 6.2739e-04
Validation auc = 0.716901
Epoch 280
Validation auc = 0.703286
Epoch 281
Validation auc = 0.689671
Epoch 282
Validation auc = 0.689202
Epoch 283
Validation auc = 0.687324
Epoch 284
Loss = 1.1990e-01, PNorm = 105.7973, GNorm = 0.3113, lr_0 = 6.2203e-04
Validation auc = 0.685446
Epoch 285
Validation auc = 0.690610
Epoch 286
Validation auc = 0.694366
Epoch 287
Validation auc = 0.695775
Epoch 288
Validation auc = 0.695305
Epoch 289
Loss = 1.8223e-03, PNorm = 105.8789, GNorm = 0.1456, lr_0 = 6.1671e-04
Validation auc = 0.694836
Epoch 290
Validation auc = 0.692488
Epoch 291
Validation auc = 0.689202
Epoch 292
Validation auc = 0.684507
Epoch 293
Validation auc = 0.682629
Epoch 294
Loss = 3.2731e-02, PNorm = 105.9419, GNorm = 1.3553, lr_0 = 6.1144e-04
Validation auc = 0.685915
Epoch 295
Validation auc = 0.686854
Epoch 296
Validation auc = 0.690141
Epoch 297
Validation auc = 0.696244
Epoch 298
Validation auc = 0.700469
Epoch 299
Loss = 4.5038e-03, PNorm = 105.9933, GNorm = 0.0219, lr_0 = 6.0621e-04
Validation auc = 0.700939
Model 0 best validation auc = 0.757746 on epoch 14
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.800722
Ensemble test auc = 0.800722
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=2100, out_features=2100, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.2, inplace=False)
    (7): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 18,866,401
Moving model to cuda
Epoch 0
Validation auc = 0.613146
Epoch 1
Validation auc = 0.743662
Epoch 2
Validation auc = 0.806573
Epoch 3
Validation auc = 0.754930
Epoch 4
Loss = 1.0128e+00, PNorm = 94.3527, GNorm = 11.3184, lr_0 = 6.5000e-04
Validation auc = 0.707042
Epoch 5
Validation auc = 0.630516
Epoch 6
Validation auc = 0.653052
Epoch 7
Validation auc = 0.702817
Epoch 8
Validation auc = 0.742019
Epoch 9
Loss = 8.6123e-01, PNorm = 94.6890, GNorm = 13.0106, lr_0 = 9.9743e-04
Validation auc = 0.695305
Epoch 10
Validation auc = 0.704695
Epoch 11
Validation auc = 0.709859
Epoch 12
Validation auc = 0.666667
Epoch 13
Validation auc = 0.696244
Epoch 14
Loss = 1.0804e+00, PNorm = 95.0391, GNorm = 14.4427, lr_0 = 9.8890e-04
Validation auc = 0.721127
Epoch 15
Validation auc = 0.691080
Epoch 16
Validation auc = 0.660094
Epoch 17
Validation auc = 0.668075
Epoch 18
Validation auc = 0.645070
Epoch 19
Loss = 5.1694e-01, PNorm = 95.2697, GNorm = 3.4464, lr_0 = 9.8045e-04
Validation auc = 0.648357
Epoch 20
Validation auc = 0.661502
Epoch 21
Validation auc = 0.666197
Epoch 22
Validation auc = 0.698122
Epoch 23
Validation auc = 0.703756
Epoch 24
Loss = 4.9550e-01, PNorm = 95.4114, GNorm = 10.8599, lr_0 = 9.7207e-04
Validation auc = 0.707981
Epoch 25
Validation auc = 0.728169
Epoch 26
Validation auc = 0.723944
Epoch 27
Validation auc = 0.720188
Epoch 28
Validation auc = 0.713146
Epoch 29
Loss = 3.2247e-01, PNorm = 95.5174, GNorm = 4.3961, lr_0 = 9.6376e-04
Validation auc = 0.710329
Epoch 30
Validation auc = 0.716901
Epoch 31
Validation auc = 0.720657
Epoch 32
Validation auc = 0.726291
Epoch 33
Validation auc = 0.713615
Epoch 34
Loss = 3.4073e-01, PNorm = 95.6206, GNorm = 6.5330, lr_0 = 9.5552e-04
Validation auc = 0.715023
Epoch 35
Validation auc = 0.692488
Epoch 36
Validation auc = 0.681690
Epoch 37
Validation auc = 0.677465
Epoch 38
Validation auc = 0.685915
Epoch 39
Loss = 3.0952e-01, PNorm = 95.7566, GNorm = 5.1619, lr_0 = 9.4735e-04
Validation auc = 0.701408
Epoch 40
Validation auc = 0.691549
Epoch 41
Validation auc = 0.679343
Epoch 42
Validation auc = 0.680751
Epoch 43
Validation auc = 0.681221
Epoch 44
Loss = 1.6762e-01, PNorm = 95.9249, GNorm = 2.9376, lr_0 = 9.3925e-04
Validation auc = 0.684507
Epoch 45
Validation auc = 0.694836
Epoch 46
Validation auc = 0.684977
Epoch 47
Validation auc = 0.692488
Epoch 48
Validation auc = 0.692019
Epoch 49
Loss = 2.3151e-01, PNorm = 96.1397, GNorm = 5.7283, lr_0 = 9.3122e-04
Validation auc = 0.685446
Epoch 50
Validation auc = 0.668075
Epoch 51
Validation auc = 0.672770
Epoch 52
Validation auc = 0.678873
Epoch 53
Validation auc = 0.675117
Epoch 54
Loss = 1.5678e-01, PNorm = 96.3940, GNorm = 2.4411, lr_0 = 9.2326e-04
Validation auc = 0.675117
Epoch 55
Validation auc = 0.678404
Epoch 56
Validation auc = 0.668075
Epoch 57
Validation auc = 0.676056
Epoch 58
Validation auc = 0.690141
Epoch 59
Loss = 1.3947e-01, PNorm = 96.6357, GNorm = 2.4334, lr_0 = 9.1537e-04
Validation auc = 0.698592
Epoch 60
Validation auc = 0.713146
Epoch 61
Validation auc = 0.735681
Epoch 62
Validation auc = 0.743192
Epoch 63
Validation auc = 0.738498
Epoch 64
Loss = 2.2736e-01, PNorm = 96.9002, GNorm = 3.4604, lr_0 = 9.0754e-04
Validation auc = 0.731925
Epoch 65
Validation auc = 0.744131
Epoch 66
Validation auc = 0.739437
Epoch 67
Validation auc = 0.737559
Epoch 68
Validation auc = 0.707512
Epoch 69
Loss = 1.9112e-01, PNorm = 97.1363, GNorm = 4.7154, lr_0 = 8.9978e-04
Validation auc = 0.668545
Epoch 70
Validation auc = 0.685446
Epoch 71
Validation auc = 0.704225
Epoch 72
Validation auc = 0.696244
Epoch 73
Validation auc = 0.708451
Epoch 74
Loss = 9.9371e-02, PNorm = 97.4143, GNorm = 1.3949, lr_0 = 8.9209e-04
Validation auc = 0.718310
Epoch 75
Validation auc = 0.726761
Epoch 76
Validation auc = 0.708920
Epoch 77
Validation auc = 0.675117
Epoch 78
Validation auc = 0.676995
Epoch 79
Loss = 1.8063e-01, PNorm = 97.6762, GNorm = 3.2281, lr_0 = 8.8447e-04
Validation auc = 0.698122
Epoch 80
Validation auc = 0.710798
Epoch 81
Validation auc = 0.710329
Epoch 82
Validation auc = 0.709390
Epoch 83
Validation auc = 0.703286
Epoch 84
Loss = 1.6993e-01, PNorm = 97.9487, GNorm = 2.0221, lr_0 = 8.7691e-04
Validation auc = 0.703756
Epoch 85
Validation auc = 0.703286
Epoch 86
Validation auc = 0.711268
Epoch 87
Validation auc = 0.685446
Epoch 88
Validation auc = 0.689202
Epoch 89
Loss = 7.7302e-02, PNorm = 98.2116, GNorm = 1.4148, lr_0 = 8.6941e-04
Validation auc = 0.693427
Epoch 90
Validation auc = 0.697183
Epoch 91
Validation auc = 0.697183
Epoch 92
Validation auc = 0.701408
Epoch 93
Validation auc = 0.707981
Epoch 94
Loss = 1.5196e-01, PNorm = 98.4542, GNorm = 3.8991, lr_0 = 8.6198e-04
Validation auc = 0.714085
Epoch 95
Validation auc = 0.715023
Epoch 96
Validation auc = 0.715023
Epoch 97
Validation auc = 0.714085
Epoch 98
Validation auc = 0.716432
Epoch 99
Loss = 1.1949e-01, PNorm = 98.6555, GNorm = 3.6280, lr_0 = 8.5461e-04
Validation auc = 0.709390
Epoch 100
Validation auc = 0.708920
Epoch 101
Validation auc = 0.707981
Epoch 102
Validation auc = 0.714554
Epoch 103
Validation auc = 0.723944
Epoch 104
Loss = 1.9052e-02, PNorm = 98.8209, GNorm = 0.8669, lr_0 = 8.4730e-04
Validation auc = 0.724883
Epoch 105
Validation auc = 0.722066
Epoch 106
Validation auc = 0.720188
Epoch 107
Validation auc = 0.727700
Epoch 108
Validation auc = 0.717840
Epoch 109
Loss = 7.8254e-02, PNorm = 99.0003, GNorm = 2.9342, lr_0 = 8.4006e-04
Validation auc = 0.720657
Epoch 110
Validation auc = 0.724413
Epoch 111
Validation auc = 0.721596
Epoch 112
Validation auc = 0.717840
Epoch 113
Validation auc = 0.706103
Epoch 114
Loss = 7.7070e-02, PNorm = 99.1866, GNorm = 2.5423, lr_0 = 8.3288e-04
Validation auc = 0.697183
Epoch 115
Validation auc = 0.688732
Epoch 116
Validation auc = 0.690610
Epoch 117
Validation auc = 0.703286
Epoch 118
Validation auc = 0.726761
Epoch 119
Loss = 8.7605e-02, PNorm = 99.4346, GNorm = 1.5637, lr_0 = 8.2576e-04
Validation auc = 0.734742
Epoch 120
Validation auc = 0.739437
Epoch 121
Validation auc = 0.738967
Epoch 122
Validation auc = 0.739906
Epoch 123
Validation auc = 0.734742
Epoch 124
Loss = 1.6619e-01, PNorm = 99.6816, GNorm = 4.0210, lr_0 = 8.1870e-04
Validation auc = 0.728638
Epoch 125
Validation auc = 0.716901
Epoch 126
Validation auc = 0.725352
Epoch 127
Validation auc = 0.749765
Epoch 128
Validation auc = 0.758685
Epoch 129
Loss = 1.3815e-01, PNorm = 99.8680, GNorm = 2.9260, lr_0 = 8.1170e-04
Validation auc = 0.761972
Epoch 130
Validation auc = 0.760094
Epoch 131
Validation auc = 0.767606
Epoch 132
Validation auc = 0.757746
Epoch 133
Validation auc = 0.755399
Epoch 134
Loss = 4.2487e-02, PNorm = 100.0554, GNorm = 0.9072, lr_0 = 8.0476e-04
Validation auc = 0.746479
Epoch 135
Validation auc = 0.732864
Epoch 136
Validation auc = 0.721127
Epoch 137
Validation auc = 0.715023
Epoch 138
Validation auc = 0.711268
Epoch 139
Loss = 1.2489e-01, PNorm = 100.2157, GNorm = 1.6315, lr_0 = 7.9788e-04
Validation auc = 0.712207
Epoch 140
Validation auc = 0.724413
Epoch 141
Validation auc = 0.736150
Epoch 142
Validation auc = 0.744131
Epoch 143
Validation auc = 0.741784
Epoch 144
Loss = 1.1959e-01, PNorm = 100.3955, GNorm = 3.1603, lr_0 = 7.9106e-04
Validation auc = 0.726761
Epoch 145
Validation auc = 0.708451
Epoch 146
Validation auc = 0.720657
Epoch 147
Validation auc = 0.732394
Epoch 148
Validation auc = 0.742723
Epoch 149
Loss = 2.3494e-01, PNorm = 100.6486, GNorm = 5.3954, lr_0 = 7.8430e-04
Validation auc = 0.743192
Epoch 150
Validation auc = 0.728169
Epoch 151
Validation auc = 0.718779
Epoch 152
Validation auc = 0.704225
Epoch 153
Validation auc = 0.698592
Epoch 154
Loss = 9.7505e-02, PNorm = 100.8860, GNorm = 0.7886, lr_0 = 7.7759e-04
Validation auc = 0.703756
Epoch 155
Validation auc = 0.711737
Epoch 156
Validation auc = 0.710329
Epoch 157
Validation auc = 0.705164
Epoch 158
Validation auc = 0.707512
Epoch 159
Loss = 6.3192e-02, PNorm = 101.0982, GNorm = 2.8306, lr_0 = 7.7095e-04
Validation auc = 0.699061
Epoch 160
Validation auc = 0.692488
Epoch 161
Validation auc = 0.684977
Epoch 162
Validation auc = 0.684507
Epoch 163
Validation auc = 0.705634
Epoch 164
Loss = 2.7949e-02, PNorm = 101.2883, GNorm = 1.3744, lr_0 = 7.6436e-04
Validation auc = 0.713146
Epoch 165
Validation auc = 0.722066
Epoch 166
Validation auc = 0.720657
Epoch 167
Validation auc = 0.720188
Epoch 168
Validation auc = 0.704695
Epoch 169
Loss = 7.2419e-02, PNorm = 101.4788, GNorm = 0.5025, lr_0 = 7.5782e-04
Validation auc = 0.694366
Epoch 170
Validation auc = 0.681690
Epoch 171
Validation auc = 0.669484
Epoch 172
Validation auc = 0.682629
Epoch 173
Validation auc = 0.696714
Epoch 174
Loss = 2.4351e-01, PNorm = 101.6659, GNorm = 3.7972, lr_0 = 7.5134e-04
Validation auc = 0.711737
Epoch 175
Validation auc = 0.725352
Epoch 176
Validation auc = 0.727230
Epoch 177
Validation auc = 0.722535
Epoch 178
Validation auc = 0.709859
Epoch 179
Loss = 9.3894e-02, PNorm = 101.9022, GNorm = 5.6720, lr_0 = 7.4492e-04
Validation auc = 0.701408
Epoch 180
Validation auc = 0.682629
Epoch 181
Validation auc = 0.672300
Epoch 182
Validation auc = 0.675117
Epoch 183
Validation auc = 0.677465
Epoch 184
Loss = 8.1193e-02, PNorm = 102.1319, GNorm = 1.7666, lr_0 = 7.3855e-04
Validation auc = 0.682160
Epoch 185
Validation auc = 0.699531
Epoch 186
Validation auc = 0.703756
Epoch 187
Validation auc = 0.698592
Epoch 188
Validation auc = 0.693427
Epoch 189
Loss = 2.5092e-02, PNorm = 102.3751, GNorm = 1.0428, lr_0 = 7.3224e-04
Validation auc = 0.686854
Epoch 190
Validation auc = 0.677465
Epoch 191
Validation auc = 0.681690
Epoch 192
Validation auc = 0.684038
Epoch 193
Validation auc = 0.679343
Epoch 194
Loss = 5.5130e-02, PNorm = 102.5955, GNorm = 1.1794, lr_0 = 7.2598e-04
Validation auc = 0.697183
Epoch 195
Validation auc = 0.705164
Epoch 196
Validation auc = 0.702347
Epoch 197
Validation auc = 0.697653
Epoch 198
Validation auc = 0.692019
Epoch 199
Loss = 4.0938e-02, PNorm = 102.7730, GNorm = 0.7050, lr_0 = 7.1977e-04
Validation auc = 0.676056
Epoch 200
Validation auc = 0.676526
Epoch 201
Validation auc = 0.676995
Epoch 202
Validation auc = 0.684038
Epoch 203
Validation auc = 0.694366
Epoch 204
Loss = 7.5627e-02, PNorm = 102.8958, GNorm = 1.9662, lr_0 = 7.1362e-04
Validation auc = 0.707042
Epoch 205
Validation auc = 0.710798
Epoch 206
Validation auc = 0.705634
Epoch 207
Validation auc = 0.700000
Epoch 208
Validation auc = 0.698122
Epoch 209
Loss = 2.0683e-02, PNorm = 102.9951, GNorm = 1.0271, lr_0 = 7.0752e-04
Validation auc = 0.704225
Epoch 210
Validation auc = 0.705164
Epoch 211
Validation auc = 0.700469
Epoch 212
Validation auc = 0.696714
Epoch 213
Validation auc = 0.695305
Epoch 214
Loss = 2.3602e-02, PNorm = 103.0832, GNorm = 1.4071, lr_0 = 7.0147e-04
Validation auc = 0.693427
Epoch 215
Validation auc = 0.692958
Epoch 216
Validation auc = 0.689202
Epoch 217
Validation auc = 0.694366
Epoch 218
Validation auc = 0.702347
Epoch 219
Loss = 3.6622e-02, PNorm = 103.1605, GNorm = 2.0653, lr_0 = 6.9548e-04
Validation auc = 0.708920
Epoch 220
Validation auc = 0.705634
Epoch 221
Validation auc = 0.708920
Epoch 222
Validation auc = 0.708920
Epoch 223
Validation auc = 0.713146
Epoch 224
Loss = 1.7009e-02, PNorm = 103.2616, GNorm = 0.3766, lr_0 = 6.8953e-04
Validation auc = 0.715962
Epoch 225
Validation auc = 0.705164
Epoch 226
Validation auc = 0.707981
Epoch 227
Validation auc = 0.707512
Epoch 228
Validation auc = 0.704695
Epoch 229
Loss = 3.7607e-02, PNorm = 103.3743, GNorm = 2.4792, lr_0 = 6.8364e-04
Validation auc = 0.704695
Epoch 230
Validation auc = 0.706573
Epoch 231
Validation auc = 0.706573
Epoch 232
Validation auc = 0.710329
Epoch 233
Validation auc = 0.707512
Epoch 234
Loss = 1.4083e-02, PNorm = 103.4807, GNorm = 1.3140, lr_0 = 6.7779e-04
Validation auc = 0.705634
Epoch 235
Validation auc = 0.704225
Epoch 236
Validation auc = 0.705634
Epoch 237
Validation auc = 0.701878
Epoch 238
Validation auc = 0.699531
Epoch 239
Loss = 4.4826e-02, PNorm = 103.5559, GNorm = 2.6285, lr_0 = 6.7200e-04
Validation auc = 0.702347
Epoch 240
Validation auc = 0.700000
Epoch 241
Validation auc = 0.699061
Epoch 242
Validation auc = 0.691549
Epoch 243
Validation auc = 0.681221
Epoch 244
Loss = 7.7826e-02, PNorm = 103.6728, GNorm = 3.0826, lr_0 = 6.6625e-04
Validation auc = 0.682629
Epoch 245
Validation auc = 0.686385
Epoch 246
Validation auc = 0.689202
Epoch 247
Validation auc = 0.686854
Epoch 248
Validation auc = 0.680282
Epoch 249
Loss = 9.5278e-03, PNorm = 103.7381, GNorm = 0.2351, lr_0 = 6.6056e-04
Validation auc = 0.676995
Epoch 250
Validation auc = 0.679812
Epoch 251
Validation auc = 0.682160
Epoch 252
Validation auc = 0.682160
Epoch 253
Validation auc = 0.678404
Epoch 254
Loss = 1.6332e-02, PNorm = 103.8206, GNorm = 1.2956, lr_0 = 6.5491e-04
Validation auc = 0.677934
Epoch 255
Validation auc = 0.691549
Epoch 256
Validation auc = 0.698122
Epoch 257
Validation auc = 0.696714
Epoch 258
Validation auc = 0.703756
Epoch 259
Loss = 5.9719e-03, PNorm = 103.8922, GNorm = 0.4224, lr_0 = 6.4931e-04
Validation auc = 0.698592
Epoch 260
Validation auc = 0.697183
Epoch 261
Validation auc = 0.697183
Epoch 262
Validation auc = 0.695305
Epoch 263
Validation auc = 0.694366
Epoch 264
Loss = 9.4765e-03, PNorm = 104.0060, GNorm = 0.7569, lr_0 = 6.4376e-04
Validation auc = 0.691549
Epoch 265
Validation auc = 0.693427
Epoch 266
Validation auc = 0.692019
Epoch 267
Validation auc = 0.698122
Epoch 268
Validation auc = 0.696244
Epoch 269
Loss = 7.3609e-03, PNorm = 104.1065, GNorm = 0.0573, lr_0 = 6.3826e-04
Validation auc = 0.700000
Epoch 270
Validation auc = 0.699531
Epoch 271
Validation auc = 0.696714
Epoch 272
Validation auc = 0.696244
Epoch 273
Validation auc = 0.693427
Epoch 274
Loss = 1.1160e-01, PNorm = 104.1762, GNorm = 1.8970, lr_0 = 6.3280e-04
Validation auc = 0.688263
Epoch 275
Validation auc = 0.688732
Epoch 276
Validation auc = 0.686385
Epoch 277
Validation auc = 0.685915
Epoch 278
Validation auc = 0.684507
Epoch 279
Loss = 1.3247e-02, PNorm = 104.2279, GNorm = 0.6219, lr_0 = 6.2739e-04
Validation auc = 0.686854
Epoch 280
Validation auc = 0.686385
Epoch 281
Validation auc = 0.687793
Epoch 282
Validation auc = 0.686854
Epoch 283
Validation auc = 0.686385
Epoch 284
Loss = 3.0671e-02, PNorm = 104.2902, GNorm = 0.2399, lr_0 = 6.2203e-04
Validation auc = 0.690610
Epoch 285
Validation auc = 0.690141
Epoch 286
Validation auc = 0.688263
Epoch 287
Validation auc = 0.685915
Epoch 288
Validation auc = 0.687793
Epoch 289
Loss = 1.5344e-02, PNorm = 104.3387, GNorm = 0.4498, lr_0 = 6.1671e-04
Validation auc = 0.688263
Epoch 290
Validation auc = 0.696244
Epoch 291
Validation auc = 0.699061
Epoch 292
Validation auc = 0.705164
Epoch 293
Validation auc = 0.707512
Epoch 294
Loss = 1.9553e-02, PNorm = 104.3893, GNorm = 1.2242, lr_0 = 6.1144e-04
Validation auc = 0.709859
Epoch 295
Validation auc = 0.710329
Epoch 296
Validation auc = 0.709390
Epoch 297
Validation auc = 0.709859
Epoch 298
Validation auc = 0.705164
Epoch 299
Loss = 1.3930e-02, PNorm = 104.4448, GNorm = 1.0248, lr_0 = 6.0621e-04
Validation auc = 0.702347
Model 0 best validation auc = 0.806573 on epoch 2
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.818182
Ensemble test auc = 0.818182
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=2100, out_features=2100, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.2, inplace=False)
    (7): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 18,866,401
Moving model to cuda
Epoch 0
Validation auc = 0.638028
Epoch 1
Validation auc = 0.685446
Epoch 2
Validation auc = 0.765258
Epoch 3
Validation auc = 0.721127
Epoch 4
Loss = 1.4264e+00, PNorm = 94.3475, GNorm = 30.4210, lr_0 = 6.5000e-04
Validation auc = 0.667136
Epoch 5
Validation auc = 0.691549
Epoch 6
Validation auc = 0.687324
Epoch 7
Validation auc = 0.721127
Epoch 8
Validation auc = 0.730986
Epoch 9
Loss = 9.2488e-01, PNorm = 94.6796, GNorm = 15.8706, lr_0 = 9.9743e-04
Validation auc = 0.600939
Epoch 10
Validation auc = 0.553521
Epoch 11
Validation auc = 0.654460
Epoch 12
Validation auc = 0.725822
Epoch 13
Validation auc = 0.736620
Epoch 14
Loss = 7.3880e-01, PNorm = 95.0119, GNorm = 7.3680, lr_0 = 9.8890e-04
Validation auc = 0.664319
Epoch 15
Validation auc = 0.622535
Epoch 16
Validation auc = 0.622535
Epoch 17
Validation auc = 0.650235
Epoch 18
Validation auc = 0.671362
Epoch 19
Loss = 5.7343e-01, PNorm = 95.2413, GNorm = 6.9678, lr_0 = 9.8045e-04
Validation auc = 0.712676
Epoch 20
Validation auc = 0.718310
Epoch 21
Validation auc = 0.725352
Epoch 22
Validation auc = 0.687793
Epoch 23
Validation auc = 0.670423
Epoch 24
Loss = 4.4998e-01, PNorm = 95.4070, GNorm = 6.0845, lr_0 = 9.7207e-04
Validation auc = 0.674178
Epoch 25
Validation auc = 0.678873
Epoch 26
Validation auc = 0.702817
Epoch 27
Validation auc = 0.715023
Epoch 28
Validation auc = 0.700939
Epoch 29
Loss = 4.3686e-01, PNorm = 95.5426, GNorm = 4.1308, lr_0 = 9.6376e-04
Validation auc = 0.674178
Epoch 30
Validation auc = 0.656808
Epoch 31
Validation auc = 0.661972
Epoch 32
Validation auc = 0.674648
Epoch 33
Validation auc = 0.669953
Epoch 34
Loss = 3.7993e-01, PNorm = 95.6979, GNorm = 2.8232, lr_0 = 9.5552e-04
Validation auc = 0.690610
Epoch 35
Validation auc = 0.711737
Epoch 36
Validation auc = 0.699061
Epoch 37
Validation auc = 0.692488
Epoch 38
Validation auc = 0.678873
Epoch 39
Loss = 2.7302e-01, PNorm = 95.8926, GNorm = 4.1777, lr_0 = 9.4735e-04
Validation auc = 0.684977
Epoch 40
Validation auc = 0.702347
Epoch 41
Validation auc = 0.719718
Epoch 42
Validation auc = 0.729577
Epoch 43
Validation auc = 0.717840
Epoch 44
Loss = 2.5010e-01, PNorm = 96.1234, GNorm = 6.4537, lr_0 = 9.3925e-04
Validation auc = 0.712676
Epoch 45
Validation auc = 0.716432
Epoch 46
Validation auc = 0.717371
Epoch 47
Validation auc = 0.709390
Epoch 48
Validation auc = 0.715493
Epoch 49
Loss = 1.5730e-01, PNorm = 96.3755, GNorm = 2.9682, lr_0 = 9.3122e-04
Validation auc = 0.721596
Epoch 50
Validation auc = 0.722066
Epoch 51
Validation auc = 0.718779
Epoch 52
Validation auc = 0.741315
Epoch 53
Validation auc = 0.730047
Epoch 54
Loss = 2.4494e-01, PNorm = 96.6395, GNorm = 3.2408, lr_0 = 9.2326e-04
Validation auc = 0.726761
Epoch 55
Validation auc = 0.720657
Epoch 56
Validation auc = 0.720657
Epoch 57
Validation auc = 0.715023
Epoch 58
Validation auc = 0.669953
Epoch 59
Loss = 2.6973e-01, PNorm = 96.9263, GNorm = 3.6420, lr_0 = 9.1537e-04
Validation auc = 0.676995
Epoch 60
Validation auc = 0.703286
Epoch 61
Validation auc = 0.716432
Epoch 62
Validation auc = 0.724883
Epoch 63
Validation auc = 0.717371
Epoch 64
Loss = 1.1698e-01, PNorm = 97.1520, GNorm = 2.8394, lr_0 = 9.0754e-04
Validation auc = 0.708920
Epoch 65
Validation auc = 0.700000
Epoch 66
Validation auc = 0.688732
Epoch 67
Validation auc = 0.655399
Epoch 68
Validation auc = 0.659624
Epoch 69
Loss = 2.6833e-01, PNorm = 97.4067, GNorm = 4.0680, lr_0 = 8.9978e-04
Validation auc = 0.704695
Epoch 70
Validation auc = 0.726761
Epoch 71
Validation auc = 0.734742
Epoch 72
Validation auc = 0.729577
Epoch 73
Validation auc = 0.722066
Epoch 74
Loss = 1.5866e-01, PNorm = 97.6867, GNorm = 2.2360, lr_0 = 8.9209e-04
Validation auc = 0.716901
Epoch 75
Validation auc = 0.715493
Epoch 76
Validation auc = 0.714085
Epoch 77
Validation auc = 0.719249
Epoch 78
Validation auc = 0.724413
Epoch 79
Loss = 1.7434e-01, PNorm = 97.9317, GNorm = 2.4358, lr_0 = 8.8447e-04
Validation auc = 0.727700
Epoch 80
Validation auc = 0.730047
Epoch 81
Validation auc = 0.724413
Epoch 82
Validation auc = 0.716432
Epoch 83
Validation auc = 0.711268
Epoch 84
Loss = 3.4685e-01, PNorm = 98.1733, GNorm = 4.4666, lr_0 = 8.7691e-04
Validation auc = 0.711737
Epoch 85
Validation auc = 0.689202
Epoch 86
Validation auc = 0.670423
Epoch 87
Validation auc = 0.671362
Epoch 88
Validation auc = 0.692488
Epoch 89
Loss = 4.1468e-02, PNorm = 98.4398, GNorm = 1.7716, lr_0 = 8.6941e-04
Validation auc = 0.702817
Epoch 90
Validation auc = 0.700000
Epoch 91
Validation auc = 0.685915
Epoch 92
Validation auc = 0.679343
Epoch 93
Validation auc = 0.673709
Epoch 94
Loss = 1.2721e-01, PNorm = 98.6819, GNorm = 4.1392, lr_0 = 8.6198e-04
Validation auc = 0.688263
Epoch 95
Validation auc = 0.692488
Epoch 96
Validation auc = 0.692488
Epoch 97
Validation auc = 0.678873
Epoch 98
Validation auc = 0.664789
Epoch 99
Loss = 1.0083e-01, PNorm = 98.9202, GNorm = 3.9974, lr_0 = 8.5461e-04
Validation auc = 0.665728
Epoch 100
Validation auc = 0.669953
Epoch 101
Validation auc = 0.674178
Epoch 102
Validation auc = 0.680751
Epoch 103
Validation auc = 0.686385
Epoch 104
Loss = 3.5140e-02, PNorm = 99.1897, GNorm = 1.4119, lr_0 = 8.4730e-04
Validation auc = 0.684977
Epoch 105
Validation auc = 0.699061
Epoch 106
Validation auc = 0.705634
Epoch 107
Validation auc = 0.705634
Epoch 108
Validation auc = 0.702817
Epoch 109
Loss = 1.5548e-01, PNorm = 99.4410, GNorm = 5.4957, lr_0 = 8.4006e-04
Validation auc = 0.711268
Epoch 110
Validation auc = 0.732864
Epoch 111
Validation auc = 0.746009
Epoch 112
Validation auc = 0.745070
Epoch 113
Validation auc = 0.740845
Epoch 114
Loss = 1.5888e-01, PNorm = 99.7184, GNorm = 2.7812, lr_0 = 8.3288e-04
Validation auc = 0.731925
Epoch 115
Validation auc = 0.725822
Epoch 116
Validation auc = 0.710329
Epoch 117
Validation auc = 0.707512
Epoch 118
Validation auc = 0.719718
Epoch 119
Loss = 2.9196e-02, PNorm = 99.9546, GNorm = 0.3525, lr_0 = 8.2576e-04
Validation auc = 0.728169
Epoch 120
Validation auc = 0.734742
Epoch 121
Validation auc = 0.735681
Epoch 122
Validation auc = 0.736150
Epoch 123
Validation auc = 0.738498
Epoch 124
Loss = 1.0121e-01, PNorm = 100.1435, GNorm = 3.3719, lr_0 = 8.1870e-04
Validation auc = 0.730516
Epoch 125
Validation auc = 0.719249
Epoch 126
Validation auc = 0.712676
Epoch 127
Validation auc = 0.715023
Epoch 128
Validation auc = 0.721127
Epoch 129
Loss = 6.9818e-02, PNorm = 100.3221, GNorm = 1.9387, lr_0 = 8.1170e-04
Validation auc = 0.713615
Epoch 130
Validation auc = 0.706573
Epoch 131
Validation auc = 0.700939
Epoch 132
Validation auc = 0.695775
Epoch 133
Validation auc = 0.695305
Epoch 134
Loss = 1.0548e-01, PNorm = 100.5337, GNorm = 2.9302, lr_0 = 8.0476e-04
Validation auc = 0.697653
Epoch 135
Validation auc = 0.712676
Epoch 136
Validation auc = 0.714554
Epoch 137
Validation auc = 0.720657
Epoch 138
Validation auc = 0.718779
Epoch 139
Loss = 9.5627e-02, PNorm = 100.7363, GNorm = 0.8579, lr_0 = 7.9788e-04
Validation auc = 0.712676
Epoch 140
Validation auc = 0.709859
Epoch 141
Validation auc = 0.702817
Epoch 142
Validation auc = 0.692019
Epoch 143
Validation auc = 0.688732
Epoch 144
Loss = 4.6353e-02, PNorm = 100.9234, GNorm = 2.1177, lr_0 = 7.9106e-04
Validation auc = 0.692488
Epoch 145
Validation auc = 0.699531
Epoch 146
Validation auc = 0.700000
Epoch 147
Validation auc = 0.700939
Epoch 148
Validation auc = 0.700000
Epoch 149
Loss = 1.1200e-01, PNorm = 101.1028, GNorm = 0.7317, lr_0 = 7.8430e-04
Validation auc = 0.712676
Epoch 150
Validation auc = 0.715962
Epoch 151
Validation auc = 0.720657
Epoch 152
Validation auc = 0.730516
Epoch 153
Validation auc = 0.729577
Epoch 154
Loss = 5.7554e-02, PNorm = 101.2648, GNorm = 1.5540, lr_0 = 7.7759e-04
Validation auc = 0.727230
Epoch 155
Validation auc = 0.724883
Epoch 156
Validation auc = 0.721127
Epoch 157
Validation auc = 0.715962
Epoch 158
Validation auc = 0.711737
Epoch 159
Loss = 8.6929e-02, PNorm = 101.4411, GNorm = 3.5586, lr_0 = 7.7095e-04
Validation auc = 0.708451
Epoch 160
Validation auc = 0.713146
Epoch 161
Validation auc = 0.721596
Epoch 162
Validation auc = 0.727700
Epoch 163
Validation auc = 0.735211
Epoch 164
Loss = 3.8300e-02, PNorm = 101.6391, GNorm = 2.2539, lr_0 = 7.6436e-04
Validation auc = 0.740376
Epoch 165
Validation auc = 0.732394
Epoch 166
Validation auc = 0.717840
Epoch 167
Validation auc = 0.703756
Epoch 168
Validation auc = 0.690610
Epoch 169
Loss = 2.0046e-02, PNorm = 101.8520, GNorm = 0.6542, lr_0 = 7.5782e-04
Validation auc = 0.685446
Epoch 170
Validation auc = 0.679812
Epoch 171
Validation auc = 0.685446
Epoch 172
Validation auc = 0.685915
Epoch 173
Validation auc = 0.679343
Epoch 174
Loss = 2.8664e-02, PNorm = 102.0804, GNorm = 1.2615, lr_0 = 7.5134e-04
Validation auc = 0.682629
Epoch 175
Validation auc = 0.690141
Epoch 176
Validation auc = 0.683568
Epoch 177
Validation auc = 0.700469
Epoch 178
Validation auc = 0.707981
Epoch 179
Loss = 8.4156e-02, PNorm = 102.3214, GNorm = 1.7124, lr_0 = 7.4492e-04
Validation auc = 0.716432
Epoch 180
Validation auc = 0.707981
Epoch 181
Validation auc = 0.695775
Epoch 182
Validation auc = 0.688263
Epoch 183
Validation auc = 0.688263
Epoch 184
Loss = 4.2112e-02, PNorm = 102.5737, GNorm = 1.3243, lr_0 = 7.3855e-04
Validation auc = 0.688263
Epoch 185
Validation auc = 0.683099
Epoch 186
Validation auc = 0.681221
Epoch 187
Validation auc = 0.686854
Epoch 188
Validation auc = 0.695775
Epoch 189
Loss = 1.9220e-02, PNorm = 102.7880, GNorm = 0.4925, lr_0 = 7.3224e-04
Validation auc = 0.700939
Epoch 190
Validation auc = 0.694836
Epoch 191
Validation auc = 0.691080
Epoch 192
Validation auc = 0.682629
Epoch 193
Validation auc = 0.671831
Epoch 194
Loss = 2.4150e-02, PNorm = 102.9747, GNorm = 0.7554, lr_0 = 7.2598e-04
Validation auc = 0.658685
Epoch 195
Validation auc = 0.647418
Epoch 196
Validation auc = 0.656338
Epoch 197
Validation auc = 0.663380
Epoch 198
Validation auc = 0.668075
Epoch 199
Loss = 2.1195e-01, PNorm = 103.1821, GNorm = 6.0266, lr_0 = 7.1977e-04
Validation auc = 0.667606
Epoch 200
Validation auc = 0.676995
Epoch 201
Validation auc = 0.689202
Epoch 202
Validation auc = 0.690610
Epoch 203
Validation auc = 0.690141
Epoch 204
Loss = 7.7819e-02, PNorm = 103.3613, GNorm = 2.5050, lr_0 = 7.1362e-04
Validation auc = 0.678873
Epoch 205
Validation auc = 0.671362
Epoch 206
Validation auc = 0.667606
Epoch 207
Validation auc = 0.659155
Epoch 208
Validation auc = 0.658216
Epoch 209
Loss = 1.4397e-01, PNorm = 103.5591, GNorm = 4.2553, lr_0 = 7.0752e-04
Validation auc = 0.670423
Epoch 210
Validation auc = 0.679812
Epoch 211
Validation auc = 0.681221
Epoch 212
Validation auc = 0.685915
Epoch 213
Validation auc = 0.680282
Epoch 214
Loss = 5.6458e-02, PNorm = 103.7737, GNorm = 3.3318, lr_0 = 7.0147e-04
Validation auc = 0.678404
Epoch 215
Validation auc = 0.674178
Epoch 216
Validation auc = 0.673239
Epoch 217
Validation auc = 0.682160
Epoch 218
Validation auc = 0.689671
Epoch 219
Loss = 5.2807e-02, PNorm = 103.9692, GNorm = 1.9802, lr_0 = 6.9548e-04
Validation auc = 0.692019
Epoch 220
Validation auc = 0.699531
Epoch 221
Validation auc = 0.707981
Epoch 222
Validation auc = 0.709390
Epoch 223
Validation auc = 0.706103
Epoch 224
Loss = 1.3577e-01, PNorm = 104.1094, GNorm = 4.8548, lr_0 = 6.8953e-04
Validation auc = 0.698592
Epoch 225
Validation auc = 0.689671
Epoch 226
Validation auc = 0.683568
Epoch 227
Validation auc = 0.681690
Epoch 228
Validation auc = 0.683099
Epoch 229
Loss = 3.7309e-02, PNorm = 104.2572, GNorm = 2.3753, lr_0 = 6.8364e-04
Validation auc = 0.680751
Epoch 230
Validation auc = 0.675117
Epoch 231
Validation auc = 0.670892
Epoch 232
Validation auc = 0.670423
Epoch 233
Validation auc = 0.669953
Epoch 234
Loss = 2.7630e-02, PNorm = 104.3503, GNorm = 2.1362, lr_0 = 6.7779e-04
Validation auc = 0.671362
Epoch 235
Validation auc = 0.671362
Epoch 236
Validation auc = 0.666667
Epoch 237
Validation auc = 0.661972
Epoch 238
Validation auc = 0.661033
Epoch 239
Loss = 1.1888e-02, PNorm = 104.4582, GNorm = 0.9863, lr_0 = 6.7200e-04
Validation auc = 0.670423
Epoch 240
Validation auc = 0.677465
Epoch 241
Validation auc = 0.684038
Epoch 242
Validation auc = 0.687324
Epoch 243
Validation auc = 0.692488
Epoch 244
Loss = 1.7987e-02, PNorm = 104.5947, GNorm = 0.8515, lr_0 = 6.6625e-04
Validation auc = 0.700469
Epoch 245
Validation auc = 0.712207
Epoch 246
Validation auc = 0.714085
Epoch 247
Validation auc = 0.712207
Epoch 248
Validation auc = 0.712207
Epoch 249
Loss = 1.3412e-02, PNorm = 104.7128, GNorm = 0.5732, lr_0 = 6.6056e-04
Validation auc = 0.708920
Epoch 250
Validation auc = 0.707042
Epoch 251
Validation auc = 0.700000
Epoch 252
Validation auc = 0.692958
Epoch 253
Validation auc = 0.689671
Epoch 254
Loss = 1.4823e-02, PNorm = 104.8302, GNorm = 1.4710, lr_0 = 6.5491e-04
Validation auc = 0.682160
Epoch 255
Validation auc = 0.677465
Epoch 256
Validation auc = 0.674178
Epoch 257
Validation auc = 0.679343
Epoch 258
Validation auc = 0.685446
Epoch 259
Loss = 1.4059e-02, PNorm = 104.9461, GNorm = 0.6239, lr_0 = 6.4931e-04
Validation auc = 0.687793
Epoch 260
Validation auc = 0.691080
Epoch 261
Validation auc = 0.698122
Epoch 262
Validation auc = 0.697183
Epoch 263
Validation auc = 0.700000
Epoch 264
Loss = 3.5533e-02, PNorm = 105.0320, GNorm = 2.1127, lr_0 = 6.4376e-04
Validation auc = 0.695305
Epoch 265
Validation auc = 0.697653
Epoch 266
Validation auc = 0.707512
Epoch 267
Validation auc = 0.711737
Epoch 268
Validation auc = 0.712676
Epoch 269
Loss = 7.3915e-02, PNorm = 105.1277, GNorm = 0.7093, lr_0 = 6.3826e-04
Validation auc = 0.714085
Epoch 270
Validation auc = 0.708920
Epoch 271
Validation auc = 0.692958
Epoch 272
Validation auc = 0.683099
Epoch 273
Validation auc = 0.682629
Epoch 274
Loss = 2.2233e-01, PNorm = 105.2248, GNorm = 6.5546, lr_0 = 6.3280e-04
Validation auc = 0.692958
Epoch 275
Validation auc = 0.698592
Epoch 276
Validation auc = 0.699061
Epoch 277
Validation auc = 0.690141
Epoch 278
Validation auc = 0.676526
Epoch 279
Loss = 4.0554e-02, PNorm = 105.3500, GNorm = 2.0212, lr_0 = 6.2739e-04
Validation auc = 0.659155
Epoch 280
Validation auc = 0.643192
Epoch 281
Validation auc = 0.650704
Epoch 282
Validation auc = 0.653052
Epoch 283
Validation auc = 0.648826
Epoch 284
Loss = 5.9552e-02, PNorm = 105.4756, GNorm = 2.5915, lr_0 = 6.2203e-04
Validation auc = 0.654460
Epoch 285
Validation auc = 0.658216
Epoch 286
Validation auc = 0.661972
Epoch 287
Validation auc = 0.667136
Epoch 288
Validation auc = 0.663380
Epoch 289
Loss = 8.5300e-02, PNorm = 105.5634, GNorm = 1.4239, lr_0 = 6.1671e-04
Validation auc = 0.656808
Epoch 290
Validation auc = 0.652582
Epoch 291
Validation auc = 0.652113
Epoch 292
Validation auc = 0.650704
Epoch 293
Validation auc = 0.652582
Epoch 294
Loss = 2.5698e-02, PNorm = 105.6680, GNorm = 2.1894, lr_0 = 6.1144e-04
Validation auc = 0.653991
Epoch 295
Validation auc = 0.653052
Epoch 296
Validation auc = 0.651174
Epoch 297
Validation auc = 0.648826
Epoch 298
Validation auc = 0.654930
Epoch 299
Loss = 2.6778e-02, PNorm = 105.7759, GNorm = 1.1665, lr_0 = 6.0621e-04
Validation auc = 0.656338
Model 0 best validation auc = 0.765258 on epoch 2
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.868754
Ensemble test auc = 0.868754
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=2100, out_features=2100, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.2, inplace=False)
    (7): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 18,866,401
Moving model to cuda
Epoch 0
Validation auc = 0.697653
Epoch 1
Validation auc = 0.688732
Epoch 2
Validation auc = 0.723005
Epoch 3
Validation auc = 0.732864
Epoch 4
Loss = 1.2088e+00, PNorm = 94.3475, GNorm = 27.0509, lr_0 = 6.5000e-04
Validation auc = 0.681690
Epoch 5
Validation auc = 0.633803
Epoch 6
Validation auc = 0.665728
Epoch 7
Validation auc = 0.676291
Epoch 8
Validation auc = 0.706573
Epoch 9
Loss = 1.2717e+00, PNorm = 94.6781, GNorm = 14.5881, lr_0 = 9.9743e-04
Validation auc = 0.690141
Epoch 10
Validation auc = 0.651643
Epoch 11
Validation auc = 0.667606
Epoch 12
Validation auc = 0.656338
Epoch 13
Validation auc = 0.661972
Epoch 14
Loss = 5.1046e-01, PNorm = 95.0259, GNorm = 8.4814, lr_0 = 9.8890e-04
Validation auc = 0.664789
Epoch 15
Validation auc = 0.674648
Epoch 16
Validation auc = 0.687793
Epoch 17
Validation auc = 0.674648
Epoch 18
Validation auc = 0.639437
Epoch 19
Loss = 3.7112e-01, PNorm = 95.2495, GNorm = 4.0117, lr_0 = 9.8045e-04
Validation auc = 0.646009
Epoch 20
Validation auc = 0.680282
Epoch 21
Validation auc = 0.666197
Epoch 22
Validation auc = 0.676995
Epoch 23
Validation auc = 0.674178
Epoch 24
Loss = 4.2645e-01, PNorm = 95.3892, GNorm = 6.1144, lr_0 = 9.7207e-04
Validation auc = 0.669953
Epoch 25
Validation auc = 0.672300
Epoch 26
Validation auc = 0.672300
Epoch 27
Validation auc = 0.677465
Epoch 28
Validation auc = 0.684507
Epoch 29
Loss = 2.6916e-01, PNorm = 95.4990, GNorm = 3.8607, lr_0 = 9.6376e-04
Validation auc = 0.688263
Epoch 30
Validation auc = 0.693897
Epoch 31
Validation auc = 0.705634
Epoch 32
Validation auc = 0.698592
Epoch 33
Validation auc = 0.679343
Epoch 34
Loss = 3.2585e-01, PNorm = 95.6188, GNorm = 4.4726, lr_0 = 9.5552e-04
Validation auc = 0.677465
Epoch 35
Validation auc = 0.683568
Epoch 36
Validation auc = 0.700469
Epoch 37
Validation auc = 0.698592
Epoch 38
Validation auc = 0.708451
Epoch 39
Loss = 2.3099e-01, PNorm = 95.7847, GNorm = 4.1169, lr_0 = 9.4735e-04
Validation auc = 0.707512
Epoch 40
Validation auc = 0.686385
Epoch 41
Validation auc = 0.665728
Epoch 42
Validation auc = 0.695775
Epoch 43
Validation auc = 0.715023
Epoch 44
Loss = 2.6966e-01, PNorm = 95.9828, GNorm = 2.4689, lr_0 = 9.3925e-04
Validation auc = 0.726761
Epoch 45
Validation auc = 0.722066
Epoch 46
Validation auc = 0.704225
Epoch 47
Validation auc = 0.685915
Epoch 48
Validation auc = 0.678873
Epoch 49
Loss = 2.3702e-01, PNorm = 96.2083, GNorm = 7.1304, lr_0 = 9.3122e-04
Validation auc = 0.691080
Epoch 50
Validation auc = 0.698592
Epoch 51
Validation auc = 0.683568
Epoch 52
Validation auc = 0.667606
Epoch 53
Validation auc = 0.660094
Epoch 54
Loss = 1.5806e-01, PNorm = 96.4471, GNorm = 1.7809, lr_0 = 9.2326e-04
Validation auc = 0.651174
Epoch 55
Validation auc = 0.642723
Epoch 56
Validation auc = 0.663850
Epoch 57
Validation auc = 0.697183
Epoch 58
Validation auc = 0.729577
Epoch 59
Loss = 2.1691e-01, PNorm = 96.6704, GNorm = 5.1743, lr_0 = 9.1537e-04
Validation auc = 0.749765
Epoch 60
Validation auc = 0.753521
Epoch 61
Validation auc = 0.748826
Epoch 62
Validation auc = 0.728169
Epoch 63
Validation auc = 0.668545
Epoch 64
Loss = 2.1571e-01, PNorm = 96.9381, GNorm = 2.2487, lr_0 = 9.0754e-04
Validation auc = 0.670892
Epoch 65
Validation auc = 0.675587
Epoch 66
Validation auc = 0.674648
Epoch 67
Validation auc = 0.681690
Epoch 68
Validation auc = 0.694366
Epoch 69
Loss = 3.2061e-01, PNorm = 97.1384, GNorm = 5.0342, lr_0 = 8.9978e-04
Validation auc = 0.687324
Epoch 70
Validation auc = 0.686385
Epoch 71
Validation auc = 0.680751
Epoch 72
Validation auc = 0.658685
Epoch 73
Validation auc = 0.680751
Epoch 74
Loss = 2.2292e-01, PNorm = 97.3280, GNorm = 1.8815, lr_0 = 8.9209e-04
Validation auc = 0.705634
Epoch 75
Validation auc = 0.716432
Epoch 76
Validation auc = 0.716432
Epoch 77
Validation auc = 0.715962
Epoch 78
Validation auc = 0.705634
Epoch 79
Loss = 2.0643e-01, PNorm = 97.5431, GNorm = 2.7592, lr_0 = 8.8447e-04
Validation auc = 0.702817
Epoch 80
Validation auc = 0.714554
Epoch 81
Validation auc = 0.721127
Epoch 82
Validation auc = 0.716901
Epoch 83
Validation auc = 0.720188
Epoch 84
Loss = 7.0942e-02, PNorm = 97.7560, GNorm = 1.0352, lr_0 = 8.7691e-04
Validation auc = 0.719718
Epoch 85
Validation auc = 0.717371
Epoch 86
Validation auc = 0.720188
Epoch 87
Validation auc = 0.725352
Epoch 88
Validation auc = 0.710329
Epoch 89
Loss = 1.1812e-01, PNorm = 97.9737, GNorm = 1.9031, lr_0 = 8.6941e-04
Validation auc = 0.686854
Epoch 90
Validation auc = 0.694836
Epoch 91
Validation auc = 0.701878
Epoch 92
Validation auc = 0.703286
Epoch 93
Validation auc = 0.707512
Epoch 94
Loss = 1.9361e-01, PNorm = 98.2008, GNorm = 2.7637, lr_0 = 8.6198e-04
Validation auc = 0.711737
Epoch 95
Validation auc = 0.712207
Epoch 96
Validation auc = 0.717371
Epoch 97
Validation auc = 0.723474
Epoch 98
Validation auc = 0.736150
Epoch 99
Loss = 7.6102e-02, PNorm = 98.4940, GNorm = 1.3918, lr_0 = 8.5461e-04
Validation auc = 0.744131
Epoch 100
Validation auc = 0.737559
Epoch 101
Validation auc = 0.713146
Epoch 102
Validation auc = 0.691080
Epoch 103
Validation auc = 0.700469
Epoch 104
Loss = 1.1742e-01, PNorm = 98.7648, GNorm = 2.9335, lr_0 = 8.4730e-04
Validation auc = 0.705164
Epoch 105
Validation auc = 0.711737
Epoch 106
Validation auc = 0.709390
Epoch 107
Validation auc = 0.706573
Epoch 108
Validation auc = 0.699531
Epoch 109
Loss = 8.6254e-02, PNorm = 99.0164, GNorm = 1.1848, lr_0 = 8.4006e-04
Validation auc = 0.704225
Epoch 110
Validation auc = 0.698592
Epoch 111
Validation auc = 0.697653
Epoch 112
Validation auc = 0.697653
Epoch 113
Validation auc = 0.699531
Epoch 114
Loss = 2.8206e-01, PNorm = 99.2314, GNorm = 2.8705, lr_0 = 8.3288e-04
Validation auc = 0.692958
Epoch 115
Validation auc = 0.698592
Epoch 116
Validation auc = 0.701408
Epoch 117
Validation auc = 0.708451
Epoch 118
Validation auc = 0.714085
Epoch 119
Loss = 1.0654e-01, PNorm = 99.4393, GNorm = 2.6155, lr_0 = 8.2576e-04
Validation auc = 0.708451
Epoch 120
Validation auc = 0.704695
Epoch 121
Validation auc = 0.709390
Epoch 122
Validation auc = 0.704695
Epoch 123
Validation auc = 0.711737
Epoch 124
Loss = 1.5115e-01, PNorm = 99.6471, GNorm = 2.5945, lr_0 = 8.1870e-04
Validation auc = 0.707512
Epoch 125
Validation auc = 0.715962
Epoch 126
Validation auc = 0.719718
Epoch 127
Validation auc = 0.719718
Epoch 128
Validation auc = 0.718779
Epoch 129
Loss = 4.8864e-02, PNorm = 99.8795, GNorm = 1.3220, lr_0 = 8.1170e-04
Validation auc = 0.707981
Epoch 130
Validation auc = 0.708920
Epoch 131
Validation auc = 0.700000
Epoch 132
Validation auc = 0.692019
Epoch 133
Validation auc = 0.682160
Epoch 134
Loss = 6.2549e-02, PNorm = 100.0725, GNorm = 0.6093, lr_0 = 8.0476e-04
Validation auc = 0.688263
Epoch 135
Validation auc = 0.693427
Epoch 136
Validation auc = 0.707512
Epoch 137
Validation auc = 0.709859
Epoch 138
Validation auc = 0.714085
Epoch 139
Loss = 9.2797e-02, PNorm = 100.2680, GNorm = 4.0993, lr_0 = 7.9788e-04
Validation auc = 0.713615
Epoch 140
Validation auc = 0.703756
Epoch 141
Validation auc = 0.707042
Epoch 142
Validation auc = 0.703756
Epoch 143
Validation auc = 0.708451
Epoch 144
Loss = 7.2452e-02, PNorm = 100.4657, GNorm = 3.7429, lr_0 = 7.9106e-04
Validation auc = 0.707512
Epoch 145
Validation auc = 0.709390
Epoch 146
Validation auc = 0.705164
Epoch 147
Validation auc = 0.700000
Epoch 148
Validation auc = 0.695775
Epoch 149
Loss = 4.6715e-02, PNorm = 100.6632, GNorm = 1.6765, lr_0 = 7.8430e-04
Validation auc = 0.695775
Epoch 150
Validation auc = 0.692488
Epoch 151
Validation auc = 0.692488
Epoch 152
Validation auc = 0.692488
Epoch 153
Validation auc = 0.697653
Epoch 154
Loss = 5.6647e-02, PNorm = 100.7932, GNorm = 2.1763, lr_0 = 7.7759e-04
Validation auc = 0.710798
Epoch 155
Validation auc = 0.713146
Epoch 156
Validation auc = 0.714554
Epoch 157
Validation auc = 0.716432
Epoch 158
Validation auc = 0.717371
Epoch 159
Loss = 8.7970e-02, PNorm = 100.9181, GNorm = 2.4835, lr_0 = 7.7095e-04
Validation auc = 0.712676
Epoch 160
Validation auc = 0.704225
Epoch 161
Validation auc = 0.709390
Epoch 162
Validation auc = 0.716901
Epoch 163
Validation auc = 0.720188
Epoch 164
Loss = 4.5503e-02, PNorm = 101.0196, GNorm = 1.9041, lr_0 = 7.6436e-04
Validation auc = 0.723474
Epoch 165
Validation auc = 0.710798
Epoch 166
Validation auc = 0.707981
Epoch 167
Validation auc = 0.699531
Epoch 168
Validation auc = 0.690141
Epoch 169
Loss = 3.0929e-02, PNorm = 101.1411, GNorm = 0.6174, lr_0 = 7.5782e-04
Validation auc = 0.687793
Epoch 170
Validation auc = 0.682629
Epoch 171
Validation auc = 0.687324
Epoch 172
Validation auc = 0.698592
Epoch 173
Validation auc = 0.712207
Epoch 174
Loss = 5.0651e-02, PNorm = 101.2982, GNorm = 0.6397, lr_0 = 7.5134e-04
Validation auc = 0.707042
Epoch 175
Validation auc = 0.703756
Epoch 176
Validation auc = 0.702817
Epoch 177
Validation auc = 0.704695
Epoch 178
Validation auc = 0.697183
Epoch 179
Loss = 1.7034e-02, PNorm = 101.4819, GNorm = 0.5791, lr_0 = 7.4492e-04
Validation auc = 0.688732
Epoch 180
Validation auc = 0.685446
Epoch 181
Validation auc = 0.679812
Epoch 182
Validation auc = 0.691080
Epoch 183
Validation auc = 0.702347
Epoch 184
Loss = 5.5822e-02, PNorm = 101.6504, GNorm = 4.2730, lr_0 = 7.3855e-04
Validation auc = 0.715493
Epoch 185
Validation auc = 0.713146
Epoch 186
Validation auc = 0.699061
Epoch 187
Validation auc = 0.694836
Epoch 188
Validation auc = 0.691549
Epoch 189
Loss = 6.5010e-02, PNorm = 101.8253, GNorm = 2.0838, lr_0 = 7.3224e-04
Validation auc = 0.680751
Epoch 190
Validation auc = 0.684038
Epoch 191
Validation auc = 0.686385
Epoch 192
Validation auc = 0.683099
Epoch 193
Validation auc = 0.677934
Epoch 194
Loss = 8.3397e-02, PNorm = 102.0330, GNorm = 0.9327, lr_0 = 7.2598e-04
Validation auc = 0.660563
Epoch 195
Validation auc = 0.649765
Epoch 196
Validation auc = 0.653052
Epoch 197
Validation auc = 0.655399
Epoch 198
Validation auc = 0.667136
Epoch 199
Loss = 9.5831e-02, PNorm = 102.3280, GNorm = 2.4120, lr_0 = 7.1977e-04
Validation auc = 0.684038
Epoch 200
Validation auc = 0.700469
Epoch 201
Validation auc = 0.711737
Epoch 202
Validation auc = 0.713615
Epoch 203
Validation auc = 0.710798
Epoch 204
Loss = 9.5192e-02, PNorm = 102.5869, GNorm = 1.8079, lr_0 = 7.1362e-04
Validation auc = 0.709390
Epoch 205
Validation auc = 0.707981
Epoch 206
Validation auc = 0.709390
Epoch 207
Validation auc = 0.714085
Epoch 208
Validation auc = 0.711737
Epoch 209
Loss = 1.5731e-02, PNorm = 102.7942, GNorm = 0.2354, lr_0 = 7.0752e-04
Validation auc = 0.714554
Epoch 210
Validation auc = 0.711737
Epoch 211
Validation auc = 0.715493
Epoch 212
Validation auc = 0.715493
Epoch 213
Validation auc = 0.716432
Epoch 214
Loss = 2.1332e-02, PNorm = 102.9343, GNorm = 0.8896, lr_0 = 7.0147e-04
Validation auc = 0.718779
Epoch 215
Validation auc = 0.720657
Epoch 216
Validation auc = 0.717371
Epoch 217
Validation auc = 0.711737
Epoch 218
Validation auc = 0.710798
Epoch 219
Loss = 1.1172e-02, PNorm = 103.0497, GNorm = 1.2789, lr_0 = 6.9548e-04
Validation auc = 0.711268
Epoch 220
Validation auc = 0.713146
Epoch 221
Validation auc = 0.703756
Epoch 222
Validation auc = 0.703286
Epoch 223
Validation auc = 0.707512
Epoch 224
Loss = 2.1408e-02, PNorm = 103.1563, GNorm = 1.6064, lr_0 = 6.8953e-04
Validation auc = 0.709390
Epoch 225
Validation auc = 0.712207
Epoch 226
Validation auc = 0.714554
Epoch 227
Validation auc = 0.718310
Epoch 228
Validation auc = 0.714085
Epoch 229
Loss = 1.4648e-02, PNorm = 103.2600, GNorm = 0.8488, lr_0 = 6.8364e-04
Validation auc = 0.707042
Epoch 230
Validation auc = 0.700000
Epoch 231
Validation auc = 0.692488
Epoch 232
Validation auc = 0.690141
Epoch 233
Validation auc = 0.684038
Epoch 234
Loss = 7.7208e-03, PNorm = 103.3676, GNorm = 0.4619, lr_0 = 6.7779e-04
Validation auc = 0.675587
Epoch 235
Validation auc = 0.670892
Epoch 236
Validation auc = 0.669953
Epoch 237
Validation auc = 0.673239
Epoch 238
Validation auc = 0.676526
Epoch 239
Loss = 5.3333e-02, PNorm = 103.4542, GNorm = 2.2267, lr_0 = 6.7200e-04
Validation auc = 0.689202
Epoch 240
Validation auc = 0.692488
Epoch 241
Validation auc = 0.694836
Epoch 242
Validation auc = 0.697183
Epoch 243
Validation auc = 0.700469
Epoch 244
Loss = 9.1547e-02, PNorm = 103.5544, GNorm = 0.0794, lr_0 = 6.6625e-04
Validation auc = 0.701878
Epoch 245
Validation auc = 0.698122
Epoch 246
Validation auc = 0.690610
Epoch 247
Validation auc = 0.687793
Epoch 248
Validation auc = 0.679812
Epoch 249
Loss = 6.8557e-02, PNorm = 103.6562, GNorm = 0.2555, lr_0 = 6.6056e-04
Validation auc = 0.675587
Epoch 250
Validation auc = 0.675587
Epoch 251
Validation auc = 0.678873
Epoch 252
Validation auc = 0.679812
Epoch 253
Validation auc = 0.687324
Epoch 254
Loss = 8.0623e-03, PNorm = 103.8002, GNorm = 0.5808, lr_0 = 6.5491e-04
Validation auc = 0.685446
Epoch 255
Validation auc = 0.688732
Epoch 256
Validation auc = 0.694836
Epoch 257
Validation auc = 0.701878
Epoch 258
Validation auc = 0.700000
Epoch 259
Loss = 1.3998e-02, PNorm = 103.9165, GNorm = 0.5577, lr_0 = 6.4931e-04
Validation auc = 0.697183
Epoch 260
Validation auc = 0.697653
Epoch 261
Validation auc = 0.703756
Epoch 262
Validation auc = 0.704225
Epoch 263
Validation auc = 0.710798
Epoch 264
Loss = 1.0870e-01, PNorm = 104.0484, GNorm = 3.3233, lr_0 = 6.4376e-04
Validation auc = 0.710798
Epoch 265
Validation auc = 0.715023
Epoch 266
Validation auc = 0.718779
Epoch 267
Validation auc = 0.712676
Epoch 268
Validation auc = 0.720188
Epoch 269
Loss = 2.8059e-02, PNorm = 104.1509, GNorm = 1.1846, lr_0 = 6.3826e-04
Validation auc = 0.717840
Epoch 270
Validation auc = 0.715493
Epoch 271
Validation auc = 0.711737
Epoch 272
Validation auc = 0.706573
Epoch 273
Validation auc = 0.701878
Epoch 274
Loss = 2.6618e-02, PNorm = 104.2581, GNorm = 1.7229, lr_0 = 6.3280e-04
Validation auc = 0.693897
Epoch 275
Validation auc = 0.696244
Epoch 276
Validation auc = 0.692019
Epoch 277
Validation auc = 0.692958
Epoch 278
Validation auc = 0.699531
Epoch 279
Loss = 5.1198e-02, PNorm = 104.3623, GNorm = 2.0439, lr_0 = 6.2739e-04
Validation auc = 0.702347
Epoch 280
Validation auc = 0.704225
Epoch 281
Validation auc = 0.701878
Epoch 282
Validation auc = 0.700939
Epoch 283
Validation auc = 0.699531
Epoch 284
Loss = 1.5294e-02, PNorm = 104.4881, GNorm = 0.5606, lr_0 = 6.2203e-04
Validation auc = 0.690141
Epoch 285
Validation auc = 0.686385
Epoch 286
Validation auc = 0.672770
Epoch 287
Validation auc = 0.664789
Epoch 288
Validation auc = 0.659155
Epoch 289
Loss = 1.1807e-01, PNorm = 104.6218, GNorm = 1.8673, lr_0 = 6.1671e-04
Validation auc = 0.649296
Epoch 290
Validation auc = 0.653991
Epoch 291
Validation auc = 0.651174
Epoch 292
Validation auc = 0.656338
Epoch 293
Validation auc = 0.661972
Epoch 294
Loss = 9.1131e-02, PNorm = 104.7356, GNorm = 1.0904, lr_0 = 6.1144e-04
Validation auc = 0.665728
Epoch 295
Validation auc = 0.673239
Epoch 296
Validation auc = 0.684038
Epoch 297
Validation auc = 0.702347
Epoch 298
Validation auc = 0.708920
Epoch 299
Loss = 4.3152e-02, PNorm = 104.8753, GNorm = 0.1745, lr_0 = 6.0621e-04
Validation auc = 0.713146
Model 0 best validation auc = 0.753521 on epoch 60
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.815774
Ensemble test auc = 0.815774
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.2, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=2100, bias=False)
        (W_h): Linear(in_features=2100, out_features=2100, bias=False)
        (W_o): Linear(in_features=2233, out_features=2100, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): Linear(in_features=2400, out_features=2100, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.2, inplace=False)
    (4): Linear(in_features=2100, out_features=2100, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.2, inplace=False)
    (7): Linear(in_features=2100, out_features=1, bias=True)
  )
)
Number of parameters = 18,866,401
Moving model to cuda
Epoch 0
Validation auc = 0.745070
Epoch 1
Validation auc = 0.673709
Epoch 2
Validation auc = 0.678404
Epoch 3
Validation auc = 0.697183
Epoch 4
Loss = 9.3759e-01, PNorm = 94.3473, GNorm = 19.2142, lr_0 = 6.5000e-04
Validation auc = 0.700000
Epoch 5
Validation auc = 0.684038
Epoch 6
Validation auc = 0.687324
Epoch 7
Validation auc = 0.654460
Epoch 8
Validation auc = 0.655399
Epoch 9
Loss = 4.1309e-01, PNorm = 94.6916, GNorm = 7.0057, lr_0 = 9.9743e-04
Validation auc = 0.669953
Epoch 10
Validation auc = 0.576526
Epoch 11
Validation auc = 0.681221
Epoch 12
Validation auc = 0.694836
Epoch 13
Validation auc = 0.734272
Epoch 14
Loss = 8.6959e-01, PNorm = 95.0400, GNorm = 18.0031, lr_0 = 9.8890e-04
Validation auc = 0.693427
Epoch 15
Validation auc = 0.681690
Epoch 16
Validation auc = 0.696714
Epoch 17
Validation auc = 0.686854
Epoch 18
Validation auc = 0.641315
Epoch 19
Loss = 5.7126e-01, PNorm = 95.2721, GNorm = 6.0325, lr_0 = 9.8045e-04
Validation auc = 0.658216
Epoch 20
Validation auc = 0.695305
Epoch 21
Validation auc = 0.702817
Epoch 22
Validation auc = 0.673239
Epoch 23
Validation auc = 0.647418
Epoch 24
Loss = 6.6135e-01, PNorm = 95.4303, GNorm = 14.0868, lr_0 = 9.7207e-04
Validation auc = 0.647418
Epoch 25
Validation auc = 0.670892
Epoch 26
Validation auc = 0.713615
Epoch 27
Validation auc = 0.712207
Epoch 28
Validation auc = 0.704225
Epoch 29
Loss = 3.1952e-01, PNorm = 95.5584, GNorm = 4.3267, lr_0 = 9.6376e-04
Validation auc = 0.702347
Epoch 30
Validation auc = 0.705634
Epoch 31
Validation auc = 0.692019
Epoch 32
Validation auc = 0.705634
Epoch 33
Validation auc = 0.714554
Epoch 34
Loss = 3.0030e-01, PNorm = 95.6940, GNorm = 4.1846, lr_0 = 9.5552e-04
Validation auc = 0.720657
Epoch 35
Validation auc = 0.723005
Epoch 36
Validation auc = 0.709859
Epoch 37
Validation auc = 0.685446
Epoch 38
Validation auc = 0.696714
Epoch 39
Loss = 2.8088e-01, PNorm = 95.8669, GNorm = 4.1616, lr_0 = 9.4735e-04
Validation auc = 0.710329
Epoch 40
Validation auc = 0.715962
Epoch 41
Validation auc = 0.712207
Epoch 42
Validation auc = 0.697183
Epoch 43
Validation auc = 0.687793
Epoch 44
Loss = 3.1862e-01, PNorm = 96.0760, GNorm = 5.8228, lr_0 = 9.3925e-04
Validation auc = 0.689671
Epoch 45
Validation auc = 0.710798
Epoch 46
Validation auc = 0.707042
Epoch 47
Validation auc = 0.715962
Epoch 48
Validation auc = 0.725822
Epoch 49
Loss = 2.3833e-01, PNorm = 96.3271, GNorm = 2.5583, lr_0 = 9.3122e-04
Validation auc = 0.712676
Epoch 50
Validation auc = 0.727230
Epoch 51
Validation auc = 0.737089
Epoch 52
Validation auc = 0.736620
Epoch 53
Validation auc = 0.726761
Epoch 54
Loss = 1.5562e-01, PNorm = 96.5536, GNorm = 3.8244, lr_0 = 9.2326e-04
Validation auc = 0.722066
Epoch 55
Validation auc = 0.721127
Epoch 56
Validation auc = 0.734742
Epoch 57
Validation auc = 0.743662
Epoch 58
Validation auc = 0.739437
Epoch 59
Loss = 1.2233e-01, PNorm = 96.7991, GNorm = 3.4153, lr_0 = 9.1537e-04
Validation auc = 0.738498
Epoch 60
Validation auc = 0.726291
Epoch 61
Validation auc = 0.717371
Epoch 62
Validation auc = 0.727230
Epoch 63
Validation auc = 0.739906
Epoch 64
Loss = 3.9856e-01, PNorm = 97.0492, GNorm = 4.0989, lr_0 = 9.0754e-04
Validation auc = 0.744131
Epoch 65
Validation auc = 0.732864
Epoch 66
Validation auc = 0.723005
Epoch 67
Validation auc = 0.710329
Epoch 68
Validation auc = 0.715023
Epoch 69
Loss = 7.9197e-02, PNorm = 97.3148, GNorm = 1.7214, lr_0 = 8.9978e-04
Validation auc = 0.705634
Epoch 70
Validation auc = 0.684507
Epoch 71
Validation auc = 0.709859
Epoch 72
Validation auc = 0.745540
Epoch 73
Validation auc = 0.762911
Epoch 74
Loss = 1.9464e-01, PNorm = 97.5860, GNorm = 3.9191, lr_0 = 8.9209e-04
Validation auc = 0.758216
Epoch 75
Validation auc = 0.752113
Epoch 76
Validation auc = 0.746479
Epoch 77
Validation auc = 0.710798
Epoch 78
Validation auc = 0.699531
Epoch 79
Loss = 1.9492e-01, PNorm = 97.9287, GNorm = 2.8136, lr_0 = 8.8447e-04
Validation auc = 0.701878
Epoch 80
Validation auc = 0.717840
Epoch 81
Validation auc = 0.725352
Epoch 82
Validation auc = 0.720188
Epoch 83
Validation auc = 0.708920
Epoch 84
Loss = 2.6732e-01, PNorm = 98.2268, GNorm = 4.8310, lr_0 = 8.7691e-04
Validation auc = 0.690610
Epoch 85
Validation auc = 0.668545
Epoch 86
Validation auc = 0.669014
Epoch 87
Validation auc = 0.676056
Epoch 88
Validation auc = 0.683099
Epoch 89
Loss = 1.8492e-01, PNorm = 98.4993, GNorm = 3.2263, lr_0 = 8.6941e-04
Validation auc = 0.698122
Epoch 90
Validation auc = 0.708920
Epoch 91
Validation auc = 0.716901
Epoch 92
Validation auc = 0.718310
Epoch 93
Validation auc = 0.726291
Epoch 94
Loss = 6.0340e-02, PNorm = 98.7644, GNorm = 1.3918, lr_0 = 8.6198e-04
Validation auc = 0.731925
Epoch 95
Validation auc = 0.733803
Epoch 96
Validation auc = 0.723005
Epoch 97
Validation auc = 0.720188
Epoch 98
Validation auc = 0.723944
Epoch 99
Loss = 9.0056e-02, PNorm = 99.0101, GNorm = 3.0420, lr_0 = 8.5461e-04
Validation auc = 0.731455
Epoch 100
Validation auc = 0.734272
Epoch 101
Validation auc = 0.737559
Epoch 102
Validation auc = 0.730516
Epoch 103
Validation auc = 0.721596
Epoch 104
Loss = 1.0128e-01, PNorm = 99.2262, GNorm = 2.6712, lr_0 = 8.4730e-04
Validation auc = 0.716901
Epoch 105
Validation auc = 0.720188
Epoch 106
Validation auc = 0.707981
Epoch 107
Validation auc = 0.693897
Epoch 108
Validation auc = 0.710798
Epoch 109
Loss = 5.3092e-02, PNorm = 99.4702, GNorm = 2.3129, lr_0 = 8.4006e-04
Validation auc = 0.724413
Epoch 110
Validation auc = 0.727700
Epoch 111
Validation auc = 0.727230
Epoch 112
Validation auc = 0.724413
Epoch 113
Validation auc = 0.720657
Epoch 114
Loss = 1.2188e-01, PNorm = 99.7067, GNorm = 1.2201, lr_0 = 8.3288e-04
Validation auc = 0.700939
Epoch 115
Validation auc = 0.691080
Epoch 116
Validation auc = 0.689671
Epoch 117
Validation auc = 0.683099
Epoch 118
Validation auc = 0.680282
Epoch 119
Loss = 8.0006e-02, PNorm = 99.9247, GNorm = 1.9916, lr_0 = 8.2576e-04
Validation auc = 0.681690
Epoch 120
Validation auc = 0.686854
Epoch 121
Validation auc = 0.696714
Epoch 122
Validation auc = 0.709390
Epoch 123
Validation auc = 0.717840
Epoch 124
Loss = 4.0868e-02, PNorm = 100.1717, GNorm = 1.2861, lr_0 = 8.1870e-04
Validation auc = 0.725822
Epoch 125
Validation auc = 0.726291
Epoch 126
Validation auc = 0.723944
Epoch 127
Validation auc = 0.721127
Epoch 128
Validation auc = 0.723474
Epoch 129
Loss = 5.1622e-02, PNorm = 100.3547, GNorm = 0.8693, lr_0 = 8.1170e-04
Validation auc = 0.729108
Epoch 130
Validation auc = 0.731455
Epoch 131
Validation auc = 0.732864
Epoch 132
Validation auc = 0.720657
Epoch 133
Validation auc = 0.716901
Epoch 134
Loss = 7.5994e-02, PNorm = 100.5212, GNorm = 3.7380, lr_0 = 8.0476e-04
Validation auc = 0.715493
Epoch 135
Validation auc = 0.717371
Epoch 136
Validation auc = 0.714085
Epoch 137
Validation auc = 0.716432
Epoch 138
Validation auc = 0.719718
Epoch 139
Loss = 3.9890e-02, PNorm = 100.7016, GNorm = 1.4340, lr_0 = 7.9788e-04
Validation auc = 0.721127
Epoch 140
Validation auc = 0.716901
Epoch 141
Validation auc = 0.715962
Epoch 142
Validation auc = 0.711268
Epoch 143
Validation auc = 0.706573
Epoch 144
Loss = 4.8552e-02, PNorm = 100.8518, GNorm = 1.2377, lr_0 = 7.9106e-04
Validation auc = 0.718779
Epoch 145
Validation auc = 0.717371
Epoch 146
Validation auc = 0.718779
Epoch 147
Validation auc = 0.721127
Epoch 148
Validation auc = 0.723944
Epoch 149
Loss = 4.8853e-02, PNorm = 100.9881, GNorm = 0.8460, lr_0 = 7.8430e-04
Validation auc = 0.725822
Epoch 150
Validation auc = 0.716901
Epoch 151
Validation auc = 0.702817
Epoch 152
Validation auc = 0.695775
Epoch 153
Validation auc = 0.699531
Epoch 154
Loss = 9.3993e-02, PNorm = 101.1746, GNorm = 3.5382, lr_0 = 7.7759e-04
Validation auc = 0.700000
Epoch 155
Validation auc = 0.699061
Epoch 156
Validation auc = 0.693427
Epoch 157
Validation auc = 0.692958
Epoch 158
Validation auc = 0.698122
Epoch 159
Loss = 9.7147e-02, PNorm = 101.3769, GNorm = 3.3824, lr_0 = 7.7095e-04
Validation auc = 0.698122
Epoch 160
Validation auc = 0.705164
Epoch 161
Validation auc = 0.709859
Epoch 162
Validation auc = 0.707042
Epoch 163
Validation auc = 0.706103
Epoch 164
Loss = 3.3885e-02, PNorm = 101.5825, GNorm = 1.3121, lr_0 = 7.6436e-04
Validation auc = 0.703286
Epoch 165
Validation auc = 0.697183
Epoch 166
Validation auc = 0.715023
Epoch 167
Validation auc = 0.729577
Epoch 168
Validation auc = 0.737559
Epoch 169
Loss = 6.0657e-02, PNorm = 101.7797, GNorm = 2.4436, lr_0 = 7.5782e-04
Validation auc = 0.740845
Epoch 170
Validation auc = 0.712676
Epoch 171
Validation auc = 0.696714
Epoch 172
Validation auc = 0.671362
Epoch 173
Validation auc = 0.665728
Epoch 174
Loss = 6.7835e-02, PNorm = 102.0512, GNorm = 2.0938, lr_0 = 7.5134e-04
Validation auc = 0.653991
Epoch 175
Validation auc = 0.667136
Epoch 176
Validation auc = 0.686385
Epoch 177
Validation auc = 0.699531
Epoch 178
Validation auc = 0.707042
Epoch 179
Loss = 5.6669e-02, PNorm = 102.3176, GNorm = 2.1547, lr_0 = 7.4492e-04
Validation auc = 0.715962
Epoch 180
Validation auc = 0.713615
Epoch 181
Validation auc = 0.708451
Epoch 182
Validation auc = 0.700939
Epoch 183
Validation auc = 0.694366
Epoch 184
Loss = 1.8153e-01, PNorm = 102.5232, GNorm = 3.8803, lr_0 = 7.3855e-04
Validation auc = 0.687324
Epoch 185
Validation auc = 0.694366
Epoch 186
Validation auc = 0.705634
Epoch 187
Validation auc = 0.714554
Epoch 188
Validation auc = 0.704695
Epoch 189
Loss = 5.5542e-02, PNorm = 102.6664, GNorm = 0.8149, lr_0 = 7.3224e-04
Validation auc = 0.701408
Epoch 190
Validation auc = 0.700000
Epoch 191
Validation auc = 0.693897
Epoch 192
Validation auc = 0.688732
Epoch 193
Validation auc = 0.687324
Epoch 194
Loss = 1.6625e-02, PNorm = 102.8279, GNorm = 0.5895, lr_0 = 7.2598e-04
Validation auc = 0.678404
Epoch 195
Validation auc = 0.684507
Epoch 196
Validation auc = 0.682629
Epoch 197
Validation auc = 0.692019
Epoch 198
Validation auc = 0.688263
Epoch 199
Loss = 8.8873e-02, PNorm = 102.9871, GNorm = 4.3674, lr_0 = 7.1977e-04
Validation auc = 0.688732
Epoch 200
Validation auc = 0.679812
Epoch 201
Validation auc = 0.674178
Epoch 202
Validation auc = 0.681221
Epoch 203
Validation auc = 0.683099
Epoch 204
Loss = 1.0790e-01, PNorm = 103.1261, GNorm = 2.4264, lr_0 = 7.1362e-04
Validation auc = 0.677934
Epoch 205
Validation auc = 0.675117
Epoch 206
Validation auc = 0.669484
Epoch 207
Validation auc = 0.668075
Epoch 208
Validation auc = 0.650704
Epoch 209
Loss = 4.7604e-02, PNorm = 103.3461, GNorm = 2.1671, lr_0 = 7.0752e-04
Validation auc = 0.647887
Epoch 210
Validation auc = 0.660563
Epoch 211
Validation auc = 0.676526
Epoch 212
Validation auc = 0.700000
Epoch 213
Validation auc = 0.715023
Epoch 214
Loss = 5.9338e-02, PNorm = 103.5445, GNorm = 2.2530, lr_0 = 7.0147e-04
Validation auc = 0.720657
Epoch 215
Validation auc = 0.724883
Epoch 216
Validation auc = 0.717840
Epoch 217
Validation auc = 0.713146
Epoch 218
Validation auc = 0.713615
Epoch 219
Loss = 1.3675e-01, PNorm = 103.7698, GNorm = 3.3249, lr_0 = 6.9548e-04
Validation auc = 0.703286
Epoch 220
Validation auc = 0.686854
Epoch 221
Validation auc = 0.684038
Epoch 222
Validation auc = 0.680751
Epoch 223
Validation auc = 0.683568
Epoch 224
Loss = 1.5949e-01, PNorm = 103.9516, GNorm = 1.8728, lr_0 = 6.8953e-04
Validation auc = 0.694366
Epoch 225
Validation auc = 0.698592
Epoch 226
Validation auc = 0.700469
Epoch 227
Validation auc = 0.705634
Epoch 228
Validation auc = 0.710798
Epoch 229
Loss = 1.1882e-01, PNorm = 104.0981, GNorm = 2.9373, lr_0 = 6.8364e-04
Validation auc = 0.712676
Epoch 230
Validation auc = 0.710798
Epoch 231
Validation auc = 0.715962
Epoch 232
Validation auc = 0.716901
Epoch 233
Validation auc = 0.710329
Epoch 234
Loss = 3.5372e-02, PNorm = 104.2585, GNorm = 1.0218, lr_0 = 6.7779e-04
Validation auc = 0.713615
Epoch 235
Validation auc = 0.710329
Epoch 236
Validation auc = 0.700469
Epoch 237
Validation auc = 0.696244
Epoch 238
Validation auc = 0.692958
Epoch 239
Loss = 3.0929e-02, PNorm = 104.3812, GNorm = 1.8782, lr_0 = 6.7200e-04
Validation auc = 0.680282
Epoch 240
Validation auc = 0.670892
Epoch 241
Validation auc = 0.672770
Epoch 242
Validation auc = 0.684507
Epoch 243
Validation auc = 0.697653
Epoch 244
Loss = 4.3901e-02, PNorm = 104.5177, GNorm = 0.7217, lr_0 = 6.6625e-04
Validation auc = 0.702817
Epoch 245
Validation auc = 0.709390
Epoch 246
Validation auc = 0.715493
Epoch 247
Validation auc = 0.721596
Epoch 248
Validation auc = 0.721127
Epoch 249
Loss = 5.3361e-02, PNorm = 104.6308, GNorm = 2.1762, lr_0 = 6.6056e-04
Validation auc = 0.723944
Epoch 250
Validation auc = 0.722535
Epoch 251
Validation auc = 0.720657
Epoch 252
Validation auc = 0.717840
Epoch 253
Validation auc = 0.709859
Epoch 254
Loss = 1.7834e-02, PNorm = 104.7374, GNorm = 0.5935, lr_0 = 6.5491e-04
Validation auc = 0.697183
Epoch 255
Validation auc = 0.691080
Epoch 256
Validation auc = 0.687793
Epoch 257
Validation auc = 0.683568
Epoch 258
Validation auc = 0.681690
Epoch 259
Loss = 2.5302e-02, PNorm = 104.8441, GNorm = 0.3271, lr_0 = 6.4931e-04
Validation auc = 0.681690
Epoch 260
Validation auc = 0.692019
Epoch 261
Validation auc = 0.696244
Epoch 262
Validation auc = 0.693897
Epoch 263
Validation auc = 0.691549
Epoch 264
Loss = 8.4992e-02, PNorm = 104.9545, GNorm = 5.4621, lr_0 = 6.4376e-04
Validation auc = 0.690141
Epoch 265
Validation auc = 0.680282
Epoch 266
Validation auc = 0.675587
Epoch 267
Validation auc = 0.671362
Epoch 268
Validation auc = 0.665728
Epoch 269
Loss = 2.0954e-02, PNorm = 105.0639, GNorm = 0.6242, lr_0 = 6.3826e-04
Validation auc = 0.652582
Epoch 270
Validation auc = 0.664319
Epoch 271
Validation auc = 0.684977
Epoch 272
Validation auc = 0.693427
Epoch 273
Validation auc = 0.699531
Epoch 274
Loss = 6.2102e-02, PNorm = 105.2264, GNorm = 2.6030, lr_0 = 6.3280e-04
Validation auc = 0.701408
Epoch 275
Validation auc = 0.700000
Epoch 276
Validation auc = 0.694366
Epoch 277
Validation auc = 0.686854
Epoch 278
Validation auc = 0.679343
Epoch 279
Loss = 1.0629e-02, PNorm = 105.3680, GNorm = 0.2657, lr_0 = 6.2739e-04
Validation auc = 0.679812
Epoch 280
Validation auc = 0.676526
Epoch 281
Validation auc = 0.679812
Epoch 282
Validation auc = 0.680751
Epoch 283
Validation auc = 0.688732
Epoch 284
Loss = 1.3469e-01, PNorm = 105.5031, GNorm = 3.4047, lr_0 = 6.2203e-04
Validation auc = 0.694366
Epoch 285
Validation auc = 0.692019
Epoch 286
Validation auc = 0.692019
Epoch 287
Validation auc = 0.692958
Epoch 288
Validation auc = 0.695305
Epoch 289
Loss = 3.4575e-02, PNorm = 105.6879, GNorm = 2.6287, lr_0 = 6.1671e-04
Validation auc = 0.702347
Epoch 290
Validation auc = 0.697653
Epoch 291
Validation auc = 0.694366
Epoch 292
Validation auc = 0.695305
Epoch 293
Validation auc = 0.695305
Epoch 294
Loss = 1.5792e-02, PNorm = 105.8863, GNorm = 0.8809, lr_0 = 6.1144e-04
Validation auc = 0.696714
Epoch 295
Validation auc = 0.700000
Epoch 296
Validation auc = 0.701878
Epoch 297
Validation auc = 0.702817
Epoch 298
Validation auc = 0.702817
Epoch 299
Loss = 1.0785e-02, PNorm = 106.0261, GNorm = 0.7317, lr_0 = 6.0621e-04
Validation auc = 0.700939
Model 0 best validation auc = 0.762911 on epoch 73
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.776641
Ensemble test auc = 0.776641
10-fold cross validation
	Seed 0 ==> test auc = 0.713426
	Seed 1 ==> test auc = 0.815172
	Seed 2 ==> test auc = 0.829621
	Seed 3 ==> test auc = 0.779952
	Seed 4 ==> test auc = 0.860325
	Seed 5 ==> test auc = 0.800722
	Seed 6 ==> test auc = 0.818182
	Seed 7 ==> test auc = 0.868754
	Seed 8 ==> test auc = 0.815774
	Seed 9 ==> test auc = 0.776641
Overall test auc = 0.807857 +/- 0.042284
Elapsed time = 0:08:16
